About EURODL |  Sections |  Subscription |  For Authors |  Special Issues |  Archives   Peer-reviewed articles |  Brief items |  Reviews

NEW

Best of EDEN 2018 Special issue
 

There are not any recent contributions.

There are not any recent contributions.

Archives

2020/I 2019/II 2019/I 2018/II 2018/I 2017/II 2017/I 2016/II 2016/I 2015/II

2015/I 2014/II 2014/I 2013/II 2013/I 2012/II 2012/I 2011/II 2011/I 2010/II

2010/I 2009/II 2009/I 2008/II 2008/I 2007/II 2007/I 2006/II 2006/I 2005/II

2005/I 2004/II 2004/I 2002 2001 2000 1999 1998 1997

EURODL Mailinglist

2851 subcribers
 

EURODL Visitors

back

Emotional obstacles for e-learning – a user psychological analysis

Sanna Juutinen [sanna.juutinen@jyu.fi],
Pertti Saariluoma [psa@it.jyu.fi]
Tietojenkasittelytieteiden laitos,
PL 35 (Agora),
40014 JYVASKYLAN YLIOPISTO [http://www.jyu.fi]

Abstracts

Finnish

E-oppimisen merkitys koulutuksessa on kasvanut jatkuvasti. Sekä julkisen että yksityisen sektorin organisaatiot käyttävät sitä laajasti. Epäilyjä kuitenkin on ilmaistu sen tehokkuuden osalta. Nämä ongelmat johtuvat usein ihmisen ja järjestelmän vuorovaikutuksesta, minkä vuoksi on tarpeen tutkia vuorovaikutusta e-oppimisessa. Me lähestymme näitä vuorovaikutusongelmia käyttäjäpsykologisesta näkökulmasta. Tämä tarkoittaa sitä, että käytämme psykologisia käsitteitä, menetelmiä ja teorioita ratkaistaksemme vuorovaikutusongelmia. Tässä artikkelissa haluamme korostaa, että on olemassa monia tunneperäisiä näkökulmia ihmisen ja teknologian vuorovaikutuksen prosesseille, ja siksi me analysoimme e-oppimiseen liittyviä emotionaalisia prosesseja.

Tässä artikkelissa olemme tutkineet verkko-oppimisjärjestelmää, jolla Suomen Verohallinto kouluttaa työntekijöitään. Esitimme tämän järjestelmän käyttäjille kyselyn, jolla analysoimme erilaisia emotionaalisia näkökulmia osana oppimisprosessia. Faktorianalyysin avulla huomasimme, että vastaajat voitiin jakaa kahteen pääryhmään. Ensimmäisen ryhmän jäsenet menestyivät opinnoissaan ja tunsivat ylpeyttä omasta suorituksestaan, kun taas toinen ryhmä menestyi opinnoissaan heikommin ja tunsivat negatiivisia tunteita e-oppimista kohtaan. Meidän kannalta tärkeää on, että negatiivinen ryhmä saadaan muutettua mahdollisimman positiiviseksi.

English

E-learning has constantly increased its importance in education. Both public and private organizations use it extensively. However, doubts have been expressed in regards to its effectiveness. These difficulties often have their origin in human-system interaction, thus there is the necessity to investigate interaction in e-learning. We approach these interaction problems from a user psychological point of view. This means that we apply psychological concepts, methods and theories to solve interaction problems. In this paper, we want to emphasize that there are many emotional aspects in the human-technology interaction processes, and consequently we analyze the emotional processes involved in e-learning.

Here we have studied the e-learning system used by the Finnish Tax Administration to train their employees. We presented their students with a questionnaire analyzing different emotional aspects of the learning process. By means of factor analysis we found that the respondents could be divided into two major groups. The members of the first group were successful and felt pride in their performance, while those of the second were less successful and developed shame reactions towards e-learning. From our point of view, it is essential that the negative group is able to be transformed as much as possible into a positive one.  

Keywords

e-learning, emotional usability, pride-frustration

List of topics

• Introduction
• Methodology
• Study design and procedure   
• Respondents   
• The questionnaire
• Results
• General discussion
• References

Introduction

E-learning is here to stay. It is one of the vital tools in modern teaching practices, and widely used by private and public organizations. It has a vital role in teaching teens, young adults, adults and senior citizens (Mason, 2006; Feinberg, 2006; Weller, Pegler and Mason 2005; Githens, 2007; Repetto and Trentin, 2008). It also provides opportunities for developing cross-cultural teaching programs (Liaw 2006; Selinger, 2004; Kim, 2002). This means that while e-learning is important today, it also has a very promising future as a central part of the modern Information and Communication Technology society.  

Despite its undeniable importance and successes, e-learning is far from being an unproblematic issue. There are a large number of studies which indicate different problems in making e-learning a solid part of modern teaching, these include: the cost of an e-learning training system presents a dilemma for some institutes and companies (Weller, 2004); in addition, online instructors and teachers require more knowledge about online teaching, as they must use teaching methods which differ to those used in traditional teaching (Bonk et al. 2004, Gaud 1999). The technology for e-learning still has its troubles today (Ardito, 2004, De Marsico et al. 2004). Further, there are attitude problems even among teaching staff towards implementing e-learning (Pegler, 2005). 

One important question in e-learning is regarding why it is not always liked by people. Why are individuals reluctant to participate in e-learning sessions and why do they not like them? These questions, although not applicable to all the students, apply to many (Juutinen and Saariluoma 2006). Nevertheless, it is a real problem for the future development of this mode of teaching. If a large segment of people do not like e-learning, it easily slows down the development of the field and causes divisions between people who will and can benefit from e-learning courses, and people who cannot. Thinking about the future, the mental obstacles for participating in e-learning easily result in losing important opportunities in advancing one's personal development.

One can regard the obstacles in participating in e-leaning as a technical problem and look for improvements in advancing technology. Undoubtedly, the advancement of technology is important, but before that advancement can take place, one must have a clear idea about how and why current technical practices should be improved. One can also see it as a usability problem (Oatley, 2004; Boehner, 2007; Rosson and Carroll 2002; De Villiers 2004; Yu, Chang et al. 2002). However, the analysis of the existing interaction technology is not necessarily sufficient, as the obstacles are often in the minds and emotions of people rather than in the immediate interaction modes (Juutinen and Saariluoma 2006, 2007). This is why we should know the mental reasons behind problems stemming from disliking, instead of just looking at how people interact with interfaces – even though this is also vital. Consequently, we regard the obstacles as emotional processes and apply user psychology to analyze the roots of the difficulties.

User psychology investigates the psychological preconditions for use and human-technology interaction (Leikas and Saariluoma 2008, Moran 1981, Oulasvirta & Saariluoma, 2004, 2006; Saariluoma, 2004). This means that user psychology must be able to explicate usability problems within psychological concepts and investigate them with psychological methods. Considering interaction problems, the main goal is to replace the traditional intuitive and folk psychological interaction analysis and design with scientifically justifiable and reliable methods, concepts and theories. This means, as far as interaction problems are concerned, that we can set and solve questions on psychological grounds. We can figure out, for example, how some psychological constructs such as long-term memory, limited capacity or mental imagery can improve our understanding of interaction processes (Oulasvirta and Saariluoma 2004, 2006, Saariluoma 2004, Saariluoma and Sajaniemi 1994).

The problems of emotional obstacles for e-learning constitute a psychological issue. In general, emotions apparently form a relatively primitive system, as shown by the importance of sub cortical and evolutionally primitive areas of brains for emotional processing (Rolls 1998). However, emotions have a very decisive role in human action and they are essential in forming priorities (Oatley and Johnson-Laird 1987, Parrott 2004). Emotions have a decisive role when we think of our relationship with the external world: they explain why we adopt some course of action and reject another. These actions-controlling functions are important also in human-technology, and therefore, it is essential to analyze them in detail. Emotions form a key area in user psychology.

The problems of emotional interaction are thus, in a key position when we consider the emotional obstacles of e-learning (Picard and Klein 2002; Norman 2004; Boehner, 2007; Klein, 2002; Oatley, 2004). Empirical research suggests also, that emotions have an important role in e-learning. The research has shown that e-learning systems can make the users frustrated, confused and reduce their interest in learning (Juutinen and Saariluoma 2006, 2007, Shneiderman, Alavi et al. 1995; Hara and Kling 2000; Zhang, Zhao et al. 2004; Drennan, Kennedy et al. 2005).

There are many motivations from an educational point of view, for paying attention to emotions in e-learning. Firstly, the importance of emotions in education has been convincingly demonstrated in traditional classroom teaching (Meyer, 2002; Hannula, 2006; Meyer, 2006; Weare, 2004; Patrick, Skinner et al. 1993; Weller, Pegler et al. 2004). Secondly, failures and frustration in using computers is commonplace for almost anyone who has operated a computer (Branco, Firth et al. 2005). This is reflected in technophobia where the existence of emotional problems has been empirically proven (Brosnan 1998).

We know also that e-learning requires the students to have more maturity and self-discipline, which indirectly implies that overcoming emotional obstacles is important in e-learning (Hiltz and Wellman 1997; Kumar, Kumar et al. 2001). A concrete example of this is demonstrated in a study by Shneiderman, Alavi et al. (1995). They found that students tend to become more actively involved in teaching in the electronic classroom, where new techniques of teaching are being used, than they are in traditional teaching.

Emotional processes involved in e-learning are also interesting from the user psychological point of view, as they enable us to deepen our understanding with respect to emotionality involved when using technology. One of the leading ideas in user psychology is explanation (Saariluoma 2004). We work to find explanations for interaction phenomena, solvable by psychological means. Emotional processes entail great potential in understanding and solving relevant problems in the area of human-technology interaction processes. This is why it is important to outline some kind of basic procedure regarding how explanatory analyses should be carried out in user psychology.

For the reasons presented, it is important to analyze emotional obstacles in e-learning. As a starting point and as a means of systematizing our endeavor, a model of how emotions are involved in e-learning should be created. Before we can proceed towards detailed emotional analyses, it is beneficial to have some overall framework and vision of the dominant mechanisms. One matter for consideration is that success in using the devices and programs generates pride, and failure generates frustration. Juutinen and Saariluoma (2006, 2007) investigated students participating in a university level e-learning program, where a connection between success and failure on one the hand, in addition to pride and frustration on the other, were found. As the number of the study's subjects was relatively small, we decided to look for empirical material related to a different e-learning context with a larger number of subjects. In this way, we thought that we could get a firm idea of the nature of differences between the students who like, and those who do not like, e-learning.

Methodology

Study design and procedure

The study was carried out during February and March 2008. The data was collected from the Finnish Tax Administration e-learning system using a questionnaire, including rating scale questions and open questions that were sent out by email to the employees. The questionnaires were to be completed on the Internet. The office has broad experience in systematic use of e-learning in their organizational training. Since the students are officials, they have to take the course as a part of their work, and therefore, we may assume that the students really have experience with the system that they do not like. It is often the case that the students with negative attitudes towards systems of the nature discussed in this paper do not have experience with them, as they simply do not use them.

Respondents

The respondents, 354 in all, who participated in the study, were employees of the Finnish Tax Administration. Of the participants, 320 were women and 34 men, and they were between 23 to 63 years of age.

The questionnaire

The questionnaire was divided into 5 different categories. The categories were:

1. Basic information,
2. E-learning system usability, functionality and user experiences,
3. Emotions,
4. Relationship towards new technology and computers,
5. Using a computer and network for studying.

These categories measured different areas of the e-learning experience. In the research at hand, Category 2 (E-learning system usability, functionality and user experiences) was used. The response scope is defined in Table 1. The respondents were asked about their user experiences in the e-learning system, with which they had been studying, using a 4 point scale. The scale's descriptors were: 1= completely agree, 2= somewhat agree, 3=somewhat disagree, 4=completely disagree. The participants stated their experience of using the e-learning system as a training facility. The answers are the respondents' subjective opinions regarding the usability of the e-learning system.  

Table 1. The user experience: array of responses

Instructions given during the course irritated me

Guidance given at the course was depressing

E-learning has some features that cause anxiety

E-learning courses make me frustrated

Using the e-learning courses is irritating

Starting the e-learning course scares me

Using the e-learning system was at times tiring

From time to time the courses can be interesting

Sometimes I feel like I am wrapped up in an e-learning course

E-learning studies make me satisfied

Studying e-learning courses sometimes seemed joyful

I feel proud after finishing an e-learning course

The designer of an e-learning course has taken the user into account

I feel pride using a computer program

I feel myself more competent as a person when I notice I have learned to manage a computer program

Frustration makes me easily quit using the program

I become easily frustrated if I can't use a program

 

Results

To get an overall view of the materials, we decided to use the principal axis factor analysis with a promax rotation. In this way, we could compress the results to their main features. We found a 4-factor solution the best for our purpose. In it, the main loadings were anxiety, satisfaction, frustration and pride. This is, of course, in harmony with our earlier findings (Juutinen and Saariluoma 2006, 2007). Frustration causes students to fail and dislike their studies, and therefore their willingness to learn diminishes (Juutinen and Saariluoma 2006). Pride has a contrary effect, it raises the willingness to learn and makes it easier to adapt and learn new things (Juutinen and Saariluoma 2006).

Factor analysis is employed in the study of user experience data to get a more holistic picture of the e-learning system and user experiences. In general, the factors are very clear and factor loadings high. Our analysis found 4 main factors that affect the students' learning experience. These were named as follows: 1=anxiety, 2=satisfaction, 3=pride and 4=frustration (see Table 2.)

The first factor was interpreted as anxiety. Anxiety was chosen here because it is an emotion activated by being in a learning situation. To be more accurate, we could call it 'anxiety due to emotional state' to differentiate between the state of anxiety and anxiety as a personality trait. In normal circumstances these people might not be any more anxious than any other persons, but the e-learning situation is distressing for them (Stöber and Schwarzer 2000, Spieberger 1972). In general, anxiety is fear with a definable content, and thus, this interpretation for the first factor makes sense.

The second factor was interpreted as satisfaction. In psychology, satisfaction is linked to the feeling of reaching a goal. It is a positive reaction to the situation. The connection between positive attributes and the item contents is evident; therefore, the selected interpretation is a well-grounded interpretation of the contents of factor 2.

The third factor is also positive. We interpreted it as pride, because pride means that one is satisfied with oneself. The feeling that one has been able to reach a goal is logical here.

Finally, we look at the fourth factor, which we termed as frustration. The items themselves openly express concepts such as dropping out and frustration. Frustration is the feeling that arises when one cannot reach a goal. It is thus, opposite in its valence to pride. On this premise, it is logical to think that frustration is the correct interpretation for factor 4. By connecting these factors with emotional states it is possible to further analyze the emotional processes relevant in e-learning. But, before we go forward it is essential to discuss the empirical data.

Table 2. Loadings for Principal Axis Factors using the Promax rotation.

	Factor
User experiences	1	2	3	4
Instructions given during the course irritated me	,801
Guidance given at the course was depressing	,776
E-learning has some features that cause anxiety	,764
E-learning courses make me frustrated	,659
Using the e-learning courses is irritating	,644
Starting the e-learning course scares me	,520
Using the e-learning system was at times very tiring	,513
From time to time courses can be interesting		,800
Sometimes I feel like I am wrapped up in a e-learning course		,733
E-learning studies make me satisfied		,676
Studying e-learning courses sometimes seemed joyful		,672
I feel proud after finishing an e-learning course		,534
The designer of an e-learning course has taken the user into account		,446
I feel pride using a computer program			,888
I feel myself more competent as a person when I notice I have learned to manage a computer program			,652
Frustration makes me easily quit using the program				,780
I become easily frustrated if I can't use a program				,659

 

The first and fourth factors (1=anxiety and 4=frustration) reflect the negative effect of e-learning. All the negative sides of studying in the e-learning system were loaded on these two factors. The respondents feared the commencement of courses; they were irritated by the system and got frustrated and anxious over some features in the e-learning system. The second and third factors (2=satisfaction and 3=pride) reflect the positive side of e-learning and using the e-learning system as a training tool. The learning experience was interesting: pride was felt when completing the course or learning something new, and study satisfaction was further experienced.

The key problem to investigate is whether there is any association between the 4 factors. To investigate correlations, Promax rotation was chosen. The factor correlation matrix (Table 3) shows that there are correlations between the factors. The 2 positive factors, satisfaction and pride, correlate positively with each other, and also the 2 negative factors, anxiety and frustration, correlate with each other.

Table 3. Factor Correlation Matrix (1=anxiety, 2=satisfaction, 3=pride, 4=frustration)

Factor	1	2	3	4
1	1,000	-,579	-,362	,434
2	-,579	1,000	, 535	-,170
3	-,362	,535	1,000	-,025
4	,434	-,170	-,025	1,000

 

The results indicate that the emotional experiences, or emotional states, are connected to the learning experiences. This is not surprising, but it shows that general competence leads to pride, and that feelings of angst as well as those of frustration, are associated with each other. The results implicate that success and positive emotions are associated with each other, and anxiety is associated with frustration. 

General discussion

User psychology forms a foundation for human-technology interaction design by analyzing users in interaction (Leikas and Saariluoma 2008, Oulasvirta and Saariluoma 2004, 2006, Saariluoma 2004, Saariluoma and Sajaniemi 1994). It does not presuppose any specific technologies as it focuses on the analyses of human mind in interaction. This is why it is logical to ask questions about emotional obstacles for e-learning in context.

Our goal has been to build a general psychological model of users step-by-step. As is well known, work has been based on simulation in this area for a long time (Card, Moran and Newell 1983). Nevertheless, until recently, relatively little attention has been paid towards the way emotions work in interaction (see Norman 2004). This is unfortunate, as emotions have a very central role in explaining human motivation and action, therefore, scientific research in emotional behavior is important (Abele-Brehm and Glendolla 2000, Franken 2002).

Here, we began with a relatively straightforward conceptual model. This means that we looked at human action from an emotional point of view and considered how the systems of emotions involved could explain the key behavioral issues. We needed to figure out how the contents of activated emotional states could explain the nature of associated human actions. During the course of this study, we have specifically been interested in valence, i.e., positive and negative stances towards e-learning. This means that we needed to model the difference between students who move towards positive learning experience and students who, although not drop-outs, nevertheless leave their opportunities unused.

Our empirical research illustrates that success generates pride and failure frustration. The following is a sample of answers that were given by the respondents describing their emotions while using the e-learning system: "I didn't quit any courses, but I did some of them sloppily because I was so frustrated;" "I was pleased with the fact that I managed to pass the course, and happy and proud about the new things that I had learned." This is not surprising per se, because success, in fact, normally generates pride and failure leads to frustration. However, it is important to see that it is precisely this combination that is important in analyzing global level emotional processes. This study utilized substantially diverse materials, but again, this same pattern of global emotions was found.

We might call this model the frustration-pride model (Figure 1.). This model is important as it provides us with an overall global schema, where emotions figure prominently in our investigation of interaction.

Figure 1. Pride-frustration model 

We cannot yet specify the detailed mechanisms involved in generating pride and frustration, but the model provides a good basis for further analyses.

At this stage, attention should be drawn towards some basic properties of emotions in explaining human action. The first issue is the psychological structure for encoding emotions. The present evidence suggests that there may either be a positive satisfaction-pride loop, or negative anxiety-frustration loop, which dominates students' emotional states and emotional interaction in the classroom.   

In the theory of motivation, emotions are often divided into those which are goal congruent and goal incongruent. Typically, positive emotions are seen as goal congruent, and negative as goal incongruent (Franken 2002). It is known that goal congruent emotions aid in achieving one's personal goals (Lazarus 1991). Goal incongruent emotions operate in an opposite manner. This means that the incongruent anxiety-frustration link causes students to be passive, and the positive satisfaction-pride link motivates them to be active. This connection to the general theory of motivation is important from a user psychological point of view.

In modern user psychology, the goal is to explain important interaction phenomena on the grounds of general psychological theories and understanding of the analyzed phenomena. Here, we can build an explanatory connection between our model and the psychological understanding of the relationship between emotions and motives (Lazarus 1991).  An explanatory understanding of interaction phenomena and psychological theories in user psychology is constructed via coherence between general theories and the found interaction phenomena (Saariluoma 2005).

Explanatory discourse is essential as it enables us to consider the predictive and practical consequences of the found interaction phenomena. In our case, it is essential that the negative loop is broken. This means that the students must be given targeted id in overcoming the frustration by means of offering positive success experiences. In psycho therapy, for example, this line of thought has long been a natural part of practice (Beck 1976). It is essential to search for suitable involvement practices to be able to break the negative loop of failure, frustration and situational anxiety. It is untested, so far, in regards to what the role of guidance should be. This is, for example, because the students take the same courses while their emotional stances differ fundamentally from one another.

In sum, the key point of user psychological research is to analyze psychological phenomena relevant in human technology interaction. User psychology connects these phenomena with our general understanding of the human mind, and bases predictions, as well as corrective action, on the general psychological understanding of the phenomena at hand.

References

1. Abele-Brehm, A. & Glendolla, G. (2000). Motivation und emotion. In Otto, J., Euler, H. & Mandl, H. (Eds.), Emotionspsychologie [The psychology of emotions] (pp. 297-305). Weinheim. Belz.
2. Ardito, C., Marsico, M. D., Lanzilotti, R., Levialdi, S., Roselli, T. & Rossano, V. (2004). Usability of e-learning tools, Proceedings of the working conference on Advanced visual interfaces. Gallipoli, Italy: ACM Press.
3. Alavi, M., Yoo, Y., &Vogel. D. R. (1997). Using Information Technology to Add Value to Management Education. The Academy of Management Journal, 40, 1310-1333.
4. Batson, L. & Feinberg, S. (2006). Game Designs that Enhance Motivation and Learning for Teenagers. Electronic Journal for the Integration of Technology in Education, Vol. 5, 34-43.
5. Boehner, K., DePaula, R., Dourish, P. & Sengers, P. (2007). How emotion is made and measured. Int. J. Human-Computer Studies, Vol. 65, 275-291.
6. Bonk, C. J., Wisher, R. A. & Lee, J. (2004). Moderating learner centered e-learning: Problems, solutions, benefits and implications. In: Roberts, T., S. (Ed.), Online collaborative learning: Theory and practice. Idea Group Inc (IGI), 54-85.
7. Branco, P., Firth, P., Encarnacao, M., & Bonato, P. (2005). Faces of Emotion in Human-Computer Interaction. CHI 2005, Portland, OR, USA.
8. Brosnan, M. J. (1998). Technophobia. London: Routledge.
9. Card, S., Moran, T. & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale: Erlbaum.
10. De Marsico M. & Levialdi, S. (2004). Evaluating web sites: exploiting user's expectations. Int. J. Human-Computer Studies 60, 381–416
11. De Villiers, R. (2004). Usability Evaluation of an E-Learning Tutorial: Criteria, Questions and Case Study. Proceedings of SAICSIT 2004, 284 – 291.
12. Drennan, J. & McColl-Kennedy, J. (2003). The relationship between Internet use and perceived performance in retail and professional service firms. Journal of Service Marketing.17, 295 – 311.
13. Franken, R. E. (2002). Human motivation. Belmont, CA: Wadsford.
14. Gaud, W.S. (1999). Assessing the impact of Web courses. Syllabus 13(4), 49-50
15. Githens, R.P. (2007). Older adults and e-learning: Opportunities and barriers. Quarterly Review of Distance Education, 8(4), 329-338.
16. Hannula, M. (2006). Motivation in mathematics: Goals reflected in emotions. Educational Studies in Mathematics: Affect in Mathematics Education: Exploring Theoretical Frameworks, A PME Special Issue, 63(2).
17. Hara, N., & Kling, B. (2000). Students' distress with a web-based distance education course. Information, Communication & Society, 3(4), 557-579.
18. Hiltz, S. R., & Wellman, B. (1997). Asynchronous learning networks as a virtual classroom. Communications of the ACM, 40(9), 44-49.
19. Juutinen, S. & Saariluoma, P. (2006). Some emotional Obstacles of E-learning, Digital Learning India, New Delhi, India.
20. Juutinen, S. & Saariluoma, P. (2007). Usability and emotional obstacles in adopting e-learning - A case study, IRMA International conference, Vancouver, Canada
21. Kim, K. & Bonk, C.J., (2002). Cross-cultural Comparisons of Online Collaboration. Journal of Computer-Mediated Communication, 8(1)
22. Kumar, A., Kumar, P.&Basu, S.C., Student perceptions of virtual education: An exploratory study. In Proceedings of 2001 Information Resources Management Association International Conference, Toronto, Ontario, Canada,  Idea Group Publishing, 400–403.
23. Lazarus, R. (1991). Cognition and motivation in emotion. American psychologist, 46, 352-367.
24. Leikas, J. & Saariluoma, P. (2008). Worth'and mental contents in designing for ageing citizens' form of life. Gerontechnology, 7, 305-318.
25. Liaw, M. (2006). E-learning and the Development of Intercultural Competence. Language Learning & Technology, 10, 3, 49-64.
26. Mason, R. (2006). Learning technologies for adult continuing education. Studies in Continuing Education, 28(2), 121-133
27. Meyer, D. K., & Turner, J. C. (2002). Discovering emotion in classroom motivation research. Educational Psychologist, 37, 107–114.
28. Meyer, D. K., & Turner, J. C. (2006). Re-conceptualizing Emotion and Motivation to Learn in Classroom Contexts. Educ Psychol Rev, 18, 377–390
29. Moran, M. R. (1981). Performance of learning disabled and low achieving secondary students on formal features of a paragraph writing task. Learning Disability Quarterly, 4(3), 271-279
30. Norman, D. (2004). Emotional design. New York: Basic Books.
31. Oatley, K. (2004). The bug in the salad: the uses of emotions in computer interfaces. Interacting with Computers, Human Computer Interaction in Latin America 16(4), 693-696
32. Oatley, K. & Johnson-Laird, P. (1987). Toward a cognitive theory of emotion. Cognition and Emotion, 1, 29-50
33. Oulasvirta, A. & Saariluoma, P. (2004). Long-term working memory and interrupting messages in human-computer interaction. Behaviour & Information Technology, 23, 53 - 64  
34. Oulasvirta, A. & Saariluoma, P. (2006). Surviving task interruptions: Investigating the implications of long-term working memory theory. International Journal of Human-Computer Studies, 64, 941-961  
35. Parrott, W. G. (2004). The nature of emotion. In M. Brewer and M. Hewstone (Eds.), Emotions and motivation. Oxford: Basil-Blackwell.
36. Patrick, B. C., Skinner, E.A. &Connell, J. P. (1993). What Motivates Children's Behavior and Emotion? Joint Effects of Perceived Control and Autonomy in the Academic Domain. Journal of Personality & Social Psychology. 65(4), 781-791.
37. Pegler, P. (2005). Objects and issues- a Sunday supplement view of continuing professional development in higher education. Open Learning 20(1), 51–64.
38. Picard, R. W. & Klein, J. (2002). Computers that recognize and respond to user emotion: theoretical and practical implications. Interacting with Computers, 14, 141-169
39. Repetto, M. & Trentin, G., (2008). ICT and lifelong learning for senior citizens, Journal of e-Learning and Knowledge Society 4 (1), 189 – 198.
40. Rolls, E. (1999). The brain and emotion. Oxford: OxfordUniversity Press.
41. Rosson, M. B. & Carroll, J. (2002). Usability engineering. San Francisco: Morgan Kaufman.
42. Saariluoma, P. (2004). Käyttäjäpsykologia. Helsinki, WSOY.
43. Saariluoma, P. (2005). Explanatory frameworks for interaction design. In: Pirhonen, A., Isomäki, H., Roast C. & Saariluoma, P. (Eds.), Future interaction design. London: Springer.
44. Saariluoma, P. & Sajaniemi, J. (1994). Transforming verbal descriptions into mathematical formulas in spreadsheet calculations. International Journal of Human-Computer Studies 421, 915-948.
45. Selinger, M. (2004). The cultural and pedagogical implications of global e-learning, Cambridge Journal of Education, 34(2), 213–229.
46. Shneiderman, B. (1976). Exploratory experiments in programmer behavior. International Journal of Computer and Information Sciences, 5, 123-143.
47. Spielberger, C. D. (1972). Conceptual and methodological issues in anxiety research. In C. D. Spielberger (Ed.), Anxiety: current trends in theory and research. Vol.2, New York: Academic Press.
48. Stöber, J., & Schwarzer, R. (2000). Angst. In J. H. Otto, H. A. Euler & H.
49. Mandl (Hrsg.), Emotionspsychologie. Ein Handbuch (S. 189198). Weinheim: Beltz, Psychologie Verlags Union.
50. Weare, K. (2004). Developing the Emotionally Literate School. London: Paul Chapman Publishing Ltd.
51. Weller, M. (2004). Learning objects and the e-learning cost dilemma. Open Learning 19 (3), 293-302
52. Weller, M., Pegler, C. & Mason, R., (2005). Use of innovative technologies on an e-learning course, Internet and Higher Education 8 (1), 61–71
53. Yu, F. Y., Chang, L. J., Liu, Y. H., & Chan, T. W. (2002). Learning preferences towards computerized competitive modes. Journal of Computer Assisted Learning, 18, 341-350.
54. Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F. Jr. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75-79