About EURODL |  Sections |  Subscription |  For Authors |  Special Issues |  Archives   Initiatives |  Special Issue Articles

NEW

Best of EDEN 2018 Special issue
 

There are not any recent contributions.

There are not any recent contributions.

Archives

2020/I 2019/II 2019/I 2018/II 2018/I 2017/II 2017/I 2016/II 2016/I 2015/II

2015/I 2014/II 2014/I 2013/II 2013/I 2012/II 2012/I 2011/II 2011/I 2010/II

2010/I 2009/II 2009/I 2008/II 2008/I 2007/II 2007/I 2006/II 2006/I 2005/II

2005/I 2004/II 2004/I 2002 2001 2000 1999 1998 1997

EURODL Mailinglist

2851 subcribers
 

EURODL Visitors

back

Evaluation of Serious Games, as a
Tool for Education for Sustainable Development

Georgia Liarakou, Eleni Sakka, Costas Gavrilakis, Costas Tsolakidis,
University of the Aegean, Greece

Abstract

Serious games could play a critical role as a pedagogical tool of Education for Sustainable Development, providing students with opportunities to experience the complex and abstract issues of sustainability. Digital game-based learning could support the development of important competences such as strategic and system thinking, planning and problem solving. The number of serious games on themes related to sustainability has grown exponentially over the last years. Although several criteria have been proposed for the evaluation of serious games most of them focus on aspects of usability and the general organization of the learning process. This study is aiming at: (a) developing a coherent set of criteria encompassing main pedagogical dimensions of Education for Sustainable Development, and (b) conducting a pilot evaluation on serious games related to this area.

Keywords: Education for Sustainable Development, serious games, evaluation

Introduction

Education for Sustainable Development (ESD), a contemporary field deriving from Environmental Education (EE), is a student-centered educational area which aims at motivating pupils to become actively involved in the learning process for environmental and societal issues. EE/ESD is constantly seeking for creative pedagogical approaches and methods that could attract and engage pupils into its subject.

Since the early stages of EE/ESD, ‘games’ have been proposed as an alternative instructional method (Taylor, 1983). Computer and video games attract increasing interest among teachers due to their potential to support learning. Such games are both engaging and interactive due to their built-in learning design. With a highly immersive nature that engages participants for a longer duration in a relaxed environment, these games also contribute to bridging the gap between teachers and students (Padature, 2011). Although there is some skepticism with regard to an extensive integration of such games into formal educational spaces (Annetta, 2008), Petkov & Rogers (2011) argue that educational systems need to incorporate the use of video games to accommodate the technology-dependent students of today. In fact, young people (even in many developing countries) grow up in a technology enhanced world. Internet, mobile phones and computer / video games being at the cutting edge of this reality, form a parallel world where many teenagers live in. Knol & de Vries (2010) argue that this particular age group is proficient at multitasking, prefers visual information over textual, is cross-media oriented, and is highly active on social network sites. The modern school should not underestimate this fact and ESD can take advantage of young people’s new competences and their alternative point of view.

However, given that computer games are generally considered as a means of entertainment, in which way they could be incorporated into the learning process without missing the desired learning goals and outcomes? According to Hirumi et al. (2010) a proper balance between education and entertainment is necessary to optimize game-based learning. When one of these dimensions dominates over the second, either the dramatic flow of story and the excitement of gameplay might be undermined or some key pedagogical principles might be lost.

In the last years a new category of computer games has emerged, entitled ‘Serious Games’, which are promising to achieve such a fragile balance. Serious games are one of the fastest-growing areas in immersive educational media today (Bronack, 2011). Zyda (2005) defined serious games as “a mental contest, played with a computer in accordance with specific rules, which uses entertainment to further government or corporate training, education, health, public policy, and strategic communication objectives”. However, Liukkonen (2009) argues that the term ‘serious games’ is not clearly defined and, in this regard, several taxonomies (e.g. Sawyer and Smith, 2008) have been created attempting to overcome this problem. Within this context, Breuer & Bente (2010) also offer a systematic clarification of several educational concepts which are associated or partially overlap with serious games, such as edutainment, entertainment education or e-learning, digital game-based learning etc. Furthermore, they identify a range of platforms, i.e. personal computers, gaming consoles or mobile phones for which serious games are designed.

In any case, serious games aim at providing an engaging, self-reinforcing context in which to motivate and educate the players (Kankaanranta & Neittaanmäki, 2009). Children’s experimentation, exploration, imagination and role-play constitute basic aspects of the learning dimension of these games (Meyer & Sørensen, 2009; Kirriemuir & McFarlane, 2004). Annetta et al. (2010) acknowledge that the design of a computer game allows players a way to build their knowledge and to develop new skills throughout its progression. It should be mentioned that their purpose can be set either by the designer or by the player. Serious games focus on situations of everyday life. Usually, such a game analyses an issue (a problem) into several “missions” of increasing difficulty. While involved into these missions, users develop several skills, such as analytical and spatial skills, strategic skills and insight, learning and recollection capabilities, psychomotor skills, visual selective attention etc. (Mitchell & Savill-Smith, 2003). Protopsaltis et al. (2011) extend the range of serious games’ learning benefits. They point out  that such games make users feel responsible for success according to their actions, match high-quality content, turn mistakes into learning elements, allow problem based learning and situated learning. According to Quinn & Neal (2008) “when a serious game is done effectively, it engages the learner's emotions and brain in a coherent experience that leaves them with new attitudes, understandings, and/or skills”. Within this context the modern school should consider seriously ways to integrate particular serious games into the existing curriculum or to utilize them as an effective additional learning tool. Petkov & Rogers (2011) support such an integration of serious games; that is why they propose developing serious games to align with the content of traditional instruction and, in the same time, traditional instruction methodology to focus on the learning aspects of these games

Within the realms of ESD, serious games could play a critical role, as a pedagogical tool that provides pupils with opportunities to experience situations which are impossible in the real world. Given also that the most environmental and sustainability issues are both complex and abstract, pupils need to develop critical thinking in order to understand the root causes as well as to evaluate the alternative solutions. Pupils need also to become emotionally involved into such situations so as to clarify the values at the roots of these issues. Serious games can potentially offer experience-based education that is directly associated with ESD goals, i.e. the conscious involvement of citizens into individual and collective actions towards sustainability (Liarakou & Flogaiti, 2007). Hummel et al. (2011) had promising results while examining whether collaborative scripts within serious games can foster complex learning and improve learning about certain problem situations in the workplace. Furthermore, several studies provide evidence that serious games (such as EnerCities and Darfur is Dying) can effectively contribute to awareness rising, concept construction and the development of positive attitudes towards sustainability issues (Liarakou et al., 2012; Knol & De Vries, 2011; Peng et al., 2010).

The number of ESD serious games has grown exponentially during the last years. Following that, a discussion has begun about the evaluation of such games. According to Liu & Ding (2009) these games need a suitable evaluation system, since they are strongly associated with educational goals. Several criteria have been proposed for the evaluation of serious games (Prensky, 2001; Kirriemuir & McFarlane, 2004; Mitchell & Savill-Smith, 2004; Pinelle, Wong & Stach, 2008; Liu & Ding, 2009). Most sets of criteria focus on aspects of usability and/or the general organization of the learning process. However, there are no criteria concerning specific pedagogical dimensions, especially with regard to innovative educational areas, such as ESD. This study is aiming at developing a coherent set of criteria as well as at conducting a pilot evaluation on serious games related to ESD.

Methodology

Three research questions were addressed:

1. To what extent serious games on sustainability are consistent with the main pedagogical aspects of ESD?
2. To what extent these Games promote an organized learning process?
3. What is the usability level of these Games?

A rigorous search of serious games, related to sustainability, was conducted at the start of this effort. Several keywords were used in order to retrieve games, such as ‘serious games’, ‘sustainability’, ‘environment’, ‘environmental education’, ‘education for sustainable development’. The search was finished when a critical number that could give a complete picture, i.e. the first 34 games, had been collected. These games constituted the sample of this study. Thirty two games were in English and two in Greek (the only games that were in this language). Games in other languages were excluded. In fact, most of the games were retrieved by two sites, i.e. Serious Games Classification and Games for Change.

Further, a draft set of evaluation criteria was developed. Among the criteria already set in literature, 9 of the most frequently presented and associated with organization of learning and usability, were selected and adapted for the purposes of this study. In fact, these criteria specify the second and the third research questions. The set was completed by 7 new criteria that were formed to reflect some critical pedagogical aspects of ESD. The first criterion is related to the holistic approach of sustainable development, a concept encompassing multiple interpretations and ideological perspectives. There is some consensus however that this concept represents a balance among three interrelated pillars, i.e. environmental protection, social welfare with equity and economic development, which could embody a new global and local vision. The other six criteria of this group represent some of the most important learning goals of ESD, which are associated with the cognitive and affective domains as well as with some competences, i.e. problem solving, critical thinking and active participation. Such competences have a key role in investigating and understanding the multiple causes and dimensions of sustainability issues and, further, in undertaking active role for the prevention of these issues.

All 16 criteria were then classified into a three-point scale, forming categories of varied quality. 5 researchers analysed and marked the games while, assessing the validity and reliability of the criteria, through a qualitative approach. After the pilot application, the set of criteria as well as their categories, were re-examined and corrected. The final set is summarised on Table 1, while the score of the games is presented on Table 2.

Table 1:   Criteria and evaluation scale for serious games related to Education for Sustainable Development

Criteria	2 points	1 point	0 points
C1. Holistic approach of sustainability	It incorporates all three dimensions (environmental, societal and economic).	It incorporates two dimensions (environmental and/or societal and/or economic).	It incorporates only one dimension (environmental or societal or economic).
C2. Knowledge	Knowledge is constructed by users while proceeding.	Knowledge is provided and is prerequisite to proceed.	Knowledge is provided and is auxiliary.
C3. Attitudes-Behaviours	It encourages users to develop their own attitudes/behaviours.	It promotes specific attitudes/behaviours.	It promotes no attitudes/behaviours.
C4. Values	It promotes negotiation of conflicting values.	It promotes specific values.	It promotes no values.
C5. Problem Solving	Users have to solve problems, inventing solutions by using the provided information.	Users have to solve problems, choosing among given solutions.	Users don’t have to solve problems in order to proceed.
C6. Systems thinking	The game progress depends on various interrelated factors.	The game progress depends on various non interrelated factors.	The game progress depends only on one factor.
C7. Active participation	The game progress depends always on users’ decisions.	The game progress depends sometimes on users’ decisions.	The game progress doesn’t depend on users’ decision.
C8. Game’s goals	Game’s goals are always displayed and their achievement is marked.	Game’s goals can be seen anytime by clicking a button.	Game’s goals are given only at the beginning.
C9. Rules	The rules are discovered by users during the playing, depending on the progress.	The rules are provided from the beginning.	The rules are not provided.
C10. Class-Collaboration	It promotes users’ collaboration and there are instructions for teachers supporting the use of the game into class.	It promotes users’ collaboration but there are not instructions for teachers.	It doesn’t promote users’ collaboration.
C11. Evaluation of knowledge level	There is knowledge assessment for all the tasks.	There is knowledge assessment for some tasks.	There is not knowledge assessment.
C12. Score	Users can see their score constantly or at the end of each stage.	Users can see their score only at the end of the game.	There is no score.
C13. Levels	There are different levels of increasing difficulty.	There are different levels, but not of increasing difficulty.	There are not different levels.
C14. Setting the parameters	The progress depends on the parameters, set by users.	The progress depends on some given scenaria.	The progress depends on a particular scenario.
C15. Save	It can be saved to be continued later (from the same point).	It can be paused.	It cannot be paused or saved.
C16. Representations	All representations are realistic.	There are both realistic and fictional representations.	All representations are fictional.

 

Results

With regard to games’ thematic subject it is evident that serious games for ESD are strongly influenced by the issue that dominates the international environmental agenda, i.e. climate change. In detail, climate change and the interrelated energy issue have been detected in 18 out of 34 games. The second most popular subject is urban environment (10 games). 6 games are referred to crucial issues of the developing world, while natural resources management and solid waste management are associated with 4 games each. 3 games deal with agriculture. Natural disasters, marine-costal ecosystems, forests and water-wetlands constitute the main interest of 2 games each. Finally only 1 deals with toxic substances and 1 with biodiversity.

Consistency with main dimensions of ESD

The first seven criteria are mainly associated with important dimensions of ESD. There are only 4 games (‘Stop Disasters’, ‘Climate challenge’, ‘3rd world Farmer’ and ‘Ayiti: the cost of life’) that deal clearly with all three main dimensions of sustainability, i.e. environmental protection, social welfare and equity and economic development. Out of 14 games, 9 cover the environmental and economic dimensions. 5 of them focus on environmental and societal dimensions. 16 games focus merely on the environment.

In more than half of the games (19), users can construct knowledge while playing. For example, in ‘Plan it green’ and ‘Stop Disasters!’ knowledge is constructed through the effects of users’ actions and decisions, while in other games is constructed through images, narrations and dialogues (e.g. ‘Food Force’). In 2 games users must consider the provided knowledge to proceed. The remaining 13 games provide only optional information. There are also games offering all three ways of acquiring knowledge, such as the ‘Water Alert’. With regard to attitudes/behaviours, 14 games allow users to choose among many alternative attitudes/behaviours, and 20 promote specific ones. For example the ‘Trouble Shooter’ emphasizes on energy saving by recommending users to turn off electric devices. 10 games provide users with the opportunity to perceive that there are issues involving conflicting values, such as the ‘Climate Challenge’ which confronts economic welfare with ecological justice. Moreover, 18 games promote specific values, i.e. solidarity, respect on human rights etc., while 6 games do not promote any values clearly.

With regard to competences intended by ESD, there are 9 games which directly promote problem solving. Among the most characteristic such games are the ‘Age of Ecology’ in which users have to invent ways to restore a dam which was damaged by an earthquake, and the ‘Green Planet’ which calls users to create a sustainable community. In 12 games, users have to solve problems by choosing among the proposed solutions. The ‘Energy Ville’ belongs to this category, as users have to combine among particular proposed energy sources to secure the energy needed for a town. There are also games (13) providing no problems for solution. Usually interactive games offer appropriate environment so that users can develop systems thinking. In this study, 21 games were reported to involve different interrelated and dynamic parameters referring to the environment, although there are 13 games whose progress depends only on one parameter. The ‘Ayiti: The cost of life’ belongs to the first case since it deals with health, education, well being and wealth simultaneously. On the other hand the progress of the ‘Go goat go’ depends only on farming of goats. With respect to the seventh criterion related to ESD, 16 games were found to require users’ decisions in order to proceed from stage to stage. In the ‘Stop Disasters!’ users have to check the soil and its resources before deciding where to construct a building. There are 6 games whose progress depends on users’ decisions, such as the ‘Errand Run’, although in other cases they have to answer questions only. Finally, 12 games require no decisions from users to proceed.

Promoting an organized learning process

Five criteria aim at investigating whether serious games are associated with an organized learning process, i.e. setting goals and rules, promoting a collaborative learning process, evaluation and score. When users know which goals have been achieved during play, they can better plan their choices / actions and can directly self-evaluate their progress. There are 12 games, the goals of which are constantly displayed onto monitor and the achievement of the players is marked. The ‘Operation Climate Control’ is such a game. Moreover, 10 games (e.g. the ‘Errand Run’) allow users to see the goals, by clicking a button. The remaining 12 games give the goals only at the beginning.

Rules also constitute a crucial factor for every game, since they set the limits and provide a clear framework. Furthermore, it is preferable if rules are discovered by users through a process of observation and hypothesis testing, rather than to be provided from the beginning. According to the results, 9 games belong to the first category, as they allow users to discover the rules during playing (e.g. the ‘Transform it!’), while 20 provide the rules from the beginning. In 5 games the rules are not provided.

Serious games can improve learning process in the class in various ways. Among the most important ones is to enhance collaborative learning and encourage creativity. To be used effectively by teachers, appropriate instructions must be provided. However, only 6 of the games examined provide suitable instructions that could help teachers to use them into class. A game that includes instructions regarding how pupils can collaborate and exchange ideas is the ‘Power Up’. Moreover, 13 games promote users’ collaboration even if they don’t provide specific instructions for teachers (e.g. the ‘3rd World farmer’). Finally, 15 games are not offered for collaboration into class.

Since evaluation is a constitutive element of every learning process, games with educational purpose, such as serious games, should assess acquisition of knowledge, through tests etc. However, in this study only 3 games (such as the “Operation Resilient Planet”) were found to assess knowledge level in every task and 5 games in some of their tasks. The strong majority of games (26) do not assess acquisition of knowledge. On the contrary, most of the games display the score that measures the successful completion of tasks. It has been assumed that it is pedagogically useful that the score is constantly presented on users’ monitor, since it constitutes an extrinsic motivation for users. In this regard, 21 games do so, while 11 present the score at the end of the game. Only 2 games have no score, although a relevant comment is provided at the end.

Usability

Although usability of a computer game is linked to many factors, four characteristic ones have been chosen for this study: levels, setting of parameters, saving options and representations.

Regarding games’ levels, broad differences have been reported. Although some games have many levels (‘Plan it Green’ has 45 levels) there are games with only one level. In detail, 15 games offer levels of increasing difficulty, 6 have different levels without increasing difficulty while 13 have no different levels. The importance of setting the parameters of a game consists in its potential to offer a different experience each time it is used. For instance, in the ‘Energy City’ users can choose among different city scenarios while they can set specific variables of each scenario (i.e. duration, type of problem, policies etc). 11 games offer this possibility. Some games (8), such as the ‘Transform it!’, offer a number of alternative scenarios although users have no possibilities to set specific parameters. 15 games depend merely on a particular scenario. With respect to interrupting the progress of a game, only 6 games can be saved to be continued later (from the same point) and 5 can be only stopped temporarily. The remaining 23 don’t allow the temporal or permanent saving of the current state. Finally, the representations contribute to games’ usability since they allow users to better perceive and understand the sustainability issues they deal with. In this sample, 23 games offer realistic representations (such as the ‘Energy Footprint’), while 8 offer partly some fictional (imaginary) representations (e.g. the ‘Honoloko’ and its health machine). There are 3 games in which the imagination is dominant throughout their representations. In the ‘Barrel Blaster’ there are, for example, flying barrels.

Table 2:   Games’ evaluation

Title of game	Number of criterion
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
1. Food Force	1	2	1	1	1	2	1	1	1	0	1	2	1	0	1	2
2. Plan it green	1	2	2	2	1	2	2	2	2	1	0	2	1	1	2	2
3. Stop Disasters!	2	2	2	1	1	2	2	2	2	2	0	2	2	2	0	2
4. Trouble Shooter	0	0	1	1	0	0	0	0	1	0	0	1	0	0	0	2
5. Water Alert	1	1	1	1	1	2	2	0	1	0	1	2	0	1	2	2
6. Darfur is Dying	1	0	2	1	0	2	0	2	1	0	0	1	0	0	0	2
7. Climate Challenge	2	2	2	2	1	2	2	2	1	1	0	1	1	2	0	2
8. Eco Saviors	0	0	1	1	0	0	0	0	1	0	0	2	2	0	1	1
9. Electrocity	1	2	1	2	2	2	2	1	2	2	0	2	2	2	2	2
10. Clim Way	1	2	2	2	2	2	2	1	1	1	0	1	0	2	2	2
11. Energuy	0	0	1	0	0	0	0	2	0	1	0	1	0	0	0	2
12. EnergyVille	1	2	2	2	1	2	2	2	2	1	0	2	2	2	0	2
13. Barrel Blaster	0	0	1	0	0	0	0	0	0	0	0	2	2	0	1	0
14. Honoloko	0	0	2	1	0	2	1	1	1	1	2	2	0	1	0	1
15. Hutnet Island	0	0	1	0	0	2	0	0	0	0	0	2	2	0	0	1
16. Mission Lighting	0	0	1	1	0	0	0	0	1	0	0	2	0	0	0	0
17. Operation Climate Control	1	2	2	1	1	2	2	2	1	2	0	1	2	1	0	2
18. Operation Resilient Planet	0	2	1	1	2	0	1	2	2	0	2	2	2	0	2	2
19. Errand Run	0	2	2	1	1	2	1	1	1	1	1	1	2	1	0	2
20. Transform it!	0	1	1	0	2	0	2	2	2	1	1	2	0	1	1	2
21. Energy city	1	2	2	2	2	2	2	2	1	1	0	1	2	2	0	2
22. Garbage King	0	0	1	1	0	0	0	1	0	0	0	2	2	0	0	2
23. Power Up	0	2	2	1	1	2	2	2	2	2	0	1	1	0	1	1
24. 3rd World farmer	2	2	2	2	2	2	2	0	2	1	0	1	0	2	0	2
25. Toxic Blaster	0	0	1	0	0	0	0	0	1	0	0	2	2	0	0	0
26. Earth hour	0	0	1	0	0	0	0	1	0	0	0	2	0	0	0	1
27. Age of Ecology	1	2	1	2	2	2	2	1	1	2	0	2	2	2	0	2
28. Energy Footprint	0	0	1	1	0	0	0	0	1	1	0	1	0	0	0	2
29. Ayiti: The cost of life	2	2	1	1	2	2	2	0	1	1	0	2	0	2	0	2
30. Go goat go	1	2	1	1	0	0	0	1	1	0	0	0	1	0	0	1
31. Disaster Watch	1	2	1	1	1	2	1	0	1	0	1	2	1	1	0	2
32. Copenhagen Challenge	0	2	1	1	1	0	1	2	1	0	2	2	2	1	0	1
33. Green Planet	1	0	2	2	2	2	2	0	2	1	0	0	0	2	0	1
34. Catchment Detox	1	2	2	2	1	2	2	1	1	2	0	2	2	2	2	2

 

Discussion

Serious games have the potential to become a significant tool for achieving the educational purposes of ESD. They could be used either by individuals and/or in the class to support effective learning processes. There are plenty of games related to sustainability and ESD, even though their use has not been investigated within the school context. However many criteria have been set to assess pedagogical aspects and the usability of serious games in general. In this study a new set of criteria has been proposed, focusing on ESD. These criteria were used to evaluate 34 serious games related to sustainability. Three subsets of criteria were formed, i.e. consistency with main dimensions of ESD, promotion of an organised learning process and usability.

It could be argued, that serious games tend to be consistent with ESD’s main dimensions since the mean value of the first set of criteria was 1.17. There is, however, much room for improving the quality of such games with respect to the holistic approach of ESD (i.e. covering the environmental, societal and economic dimensions of the topics), construction of knowledge, negotiation of conflicting values and promotion of problem solving. However, they sufficiently cover other pedagogical aspects of ESD, such as the encouragement of users to develop their own attitudes and behaviours, the encouragement of systems thinking and active participation regarding the issues they are dealing with. Furthermore, the mean value of the second set of criteria was 1.21, identifying that serious games could promote an organized learning process, even though they should emphasize more on the setting of goals and the evaluation of knowledge. It would be also important for these games to provide the appropriate conditions, such as giving sufficient instructions for teachers, regarding the use of the games in class. On the other hand, there are issues such as ways of providing the rules and the score, which are sufficiently covered. Finally, serious games demonstrate a quite high level of usability, i.e. mean value 1.39. Apart from the possibility to save the current status on the progress of the player, which is insufficiently provided, they usually have different levels of increasing difficulty. Also they provide users with the possibility of setting the parameters and of experiencing sustainability issues through realistic representations.

Although this study contributes to the development of a new set of criteria that would be used to evaluate serious games related to ESD, further research should be conducted to improve criteria to fully comply with all the qualitative features of ESD. Moreover, there is a great scope for improving the formulation of criteria so as to become clearer and more objective. Within the context of this study the games have been evaluated by the researchers. It would be crucial to conduct such an evaluation within a class. The views and opinions of teachers and pupils could provide an integrated understanding of how such games actually affect the learning process.

References

1. Annetta, L.A. (2008). Video games in education: Why they should be used and how they are being used. Theory into Practice, 47(3), (pp. 229–239).
2. Annetta, L.; Folta, E. and Klesath, M. (2010). V-Learning: Distance Education in the 21st Century through 3D Virtual Learning Environments. London – New York: Springer.
3. Breuer, J. and Bente, G. (2010). Why so serious? On the Relation of Serious Games and Learning. Journal for Computer Game Culture, 4(1), (pp. 7-24). Eludamos.
4. Bronack, S.C. (2011). The Role of Immersive Media in Online Education. The Journal of Continuing Higher Education, 59, (pp. 113–117).
5. Hirumi, A.; Appelman, B.; Rieber, L. and Van Eck, R. (2010). Game Design as a Collaborative Process. TechTrends, 54(5), (pp. 38-45).
6. Hummel, H.G.K.; Van Houcke, J.; Nadolski, R.J.; Van der Hiele, T.; Kurvers, H. and Löhr, A. (2011). Scripted collaboration in gaming for complex learning: Effects of multiple perspectives when acquiring water management skills. British Journal of Educational Technology, 42(6), (pp. 1029-1041).
7. Kankaanranta, M-H.; Neittaanmäki, P. (2009). Design and Use of Serious Games. New York: Springer
8. Kirriemuir, J.; McFarlane, A. (2004). Literature review in games and learning. Futurelab. Retrieved July, 10, 2012, from http://archive.futurelab.org.uk/resources/documents/lit_reviews/Games_Review.pdf
9. Knol, E. and De Vries, P.W. (2011). EnerCities, a Serious Game to Stimulate Sustainability and Energy Conservation: Preliminary Results. eLearning Papers, 25.
10. Knol, E. and De Vries, P.W. (2010). EnerCities: Educational Game about Energy. Proceedings CESB10 Central Europe towards Sustainable Building.
11. Liarakou G. and Flogaitis E. (2007). From Environmental Education to Education for Sustainable Development: Issues, Trends and Proposals. Athens: Nissos Publications
12. Liarakou, G.; Daskolia, M. and Papanikolaou, A. (2012). Children Learning about ‘Urban Sustainability’ through Playing and Re-constructing a Half-Baked Microworld. In C. Kynigos, J.E. Clayson and N. Yiannoutsou (Eds), Constructionism 2012. Theory, Practice and Impact (pp. 156-165). Athens: Educational Technology Lab, University of Athens – Vivliosynergatiki Publishing.
13. Liu, S.; Ding, W. (2009). An Approach to Evaluation Component Design in Building Serious Game. In: M. Chang, R. Kuo, Kinshuk, G.-D. Chen and M. Hiroshe, Edutainment ’09 Proceedings of 4th International Conference on E-learning and Games: Learning by Playing. Game-based Education System Design and Development, (pp. 141‑148). Berlin: Springer-Verlag
14. Liukkonen, T.N. (2009). VIPROSA – Game-like Tool for Visual Process Simulation and Analysis. In: M. Kankaanranta and P. Neittaanmäki (eds), Design and Use of Serious Games, (pp. 185-206). New York: Springer
15. Meyer, B.; Sørensen, B.H. (2009). Designing Serious Games for Computer Assisted Language Learning – a Framework for Development and Analysis. In: M. Kankaanranta and P. Neittaanmäki (eds), Design and Use of Serious Games, (pp.69-82). New York: Springer
16. Mitchell, A.; Savill-Smith, C. (2004). Τhe use of computer and video games for learning: A review of the literature. London: Learning and Skills Development Agency
17. Padature, J. (2011). Game Based Learning: Missing Connections. Elearningeuropa.info. Retrieved July, 10, 2012, from http://www.elearningeuropa.info/el/article/Game-Based-Learning%3A-Missing-Connections
18. Peng, W.; Lee, M. and Heeter, C. (2010). The Effects of a Serious Game on Role-Taking and Willingness to Help. Journal of Communication, 60, (pp. 723-742).
19. Petkov, M. and Rogers, G.E. (2011). Using Gaming to Motivate Today’s Technology-Dependent Students. Journal of sTEm Teacher Education, 48(1), (pp. 7-12).
20. Pinelle, D.; Wong, N.; Stach, T. (2008). Heuristic Evaluation for Games: Usability Principles for Video Game Design. In: M. Czerwinski and A. Lund, CHI ’08 Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (pp. 1453-1462). New York: ACM
21. Prensky, M. (2001). Digital Game-based Learning. New York: McGraw-Hill
22. Protopsaltis, A.; Panesse, L.; Pappa, D. and Hetzner, S. (2011). Serious Games and Formal and Informal Learning. eLearning Papers, 25.
23. Quinn, C.; Neal, L. (2008). Serious Games for serious topics. eLearn Magazine, 2008(3). Retrieved July, 10, 2012, from http://www.elearnmag.org/subpage.cfm?section=opinion&article=96-1
24. Sawyer, B.; Smith, R. (2008). Serious games taxonomy. Serious Games Initiative.
25. Taylor, J.L. (1983). Guide on simulation and gaming for environmental education. UNESCO-UNEP, International Environmental Education Programme – Environmental Education Series 2. Paris: UNESCO
26. Zyda, M. (2005). From Visual Simulation to Virtual Reality to Games. Computer, 38(9), (pp. 25-32). IEEE Computer society