Learning to feel: Education, affective outcomes and the use of online teaching and learning
Dr. Martha
Cleveland-Innes [martic@athabascau.ca]
Associate Professor
Athabasca University
Centre for Distance
Education
1 University Drive
Athabasca, Alberta AB T9S 3A3
Dr. Mohamed Ally [mohameda@athabascau.ca]
Associate Professor
Athabasca University
1 University Drive
Athabasca, Alberta
T9S 3A3
An earlier version of this paper was presented at the European Distance Education Network (EDEN) Research Workshop in Barcelona, Spain, October 2006.
Abstract
Research employing an experimental design pilot tested two delivery platforms, WebCT and ElluminateLive, for the generation of affective learning outcomes in the workplace. Ten different organizations across Western Canada asked their call centre/help desk staff to participate in an online course on customer service. One hundred and one participants were randomly assigned to two types of online learning management systems. Data comparing results of the two groups are inconclusive in relation to delivery outcomes, but indicate there is potential for soft skill development and affective gain using online delivery. Both groups performed well on tests of knowledge regarding appropriate affect in customer service environments. Soft skill assessment showed small gains from time one to time two for participants studying in both platforms. Differences between groups were seen in two observations. There was greater engagement and interaction among participants in the WebCT group. Additionally, the WebCT group yielded higher exam scores, but differences between exam means were not statistically significant.
Introduction
As early as the 1890s, education philosophers called for teachers to go beyond knowledge base and cognitive skill acquisition to moral and emotional development of students. This generated an increased awareness among teachers of their role as agents of socialization, and the achievement of affective outcomes, for children in their care. Later, a movement against teaching values in public education systems emerged: 1960s to present day. Our current social climate of rapid economic restructuring, globalization, and increasing technological sophistication requires an urbane level of moral agency and human relationship skills, with appropriate attitudes toward diversity. These attributes and skills are fostered through affective learning outcomes.
We submit that a return to designing instruction for affective outcomes is timely and appropriate. Structures related to human social behavior fall under the domain of affective outcomes in education, underrepresented in curriculum documents, curriculum planning, and in learner assessment (Claxton, 1989; Postman, 1997). In current education designs, student cognitive development is the essential outcome, leaving the affective domain behind. Others argue, however, that interpretations of quality and effectiveness in education that does not include affective outcomes are inadequate as measures of educational outcomes. (Leonard, Bourke & Schofield, 2004).
Education is currently experiencing a "radical and far-reaching restructuring process …" (Peters, 2002, p. 26). Affective educational outcomes, defined as learning outcomes that focus on "individual disposition, willingness, preferences, enjoyments …..." (Gronlund, 2000, p. 57) can be reintegrated as a critical focus during this restructuring. Evidence that such outcomes are lacking, but required, in education can be found in the soft-skill shortage among employees in the workplace (Clark, 2005; MacLeod, 2000). Soft skills are developed in the affective domain and are important to productivity, employee satisfaction and health and ultimately, economic success for society. Soft skills include self-awareness, analytical thinking, leadership skills, team-building skills, flexibility, the ability to communicate effectively, creativity, problem-solving skills, listening skills, diplomacy and change-readiness.
A shortage of these skills has been identified in both Canada and the U.K. Human Resource and Skill Development Canada identified, for example, that employees in call centre/help desk environments must be able to handle not only the technological aspects of their job, but a variety of customer queries in an appropriate manner, using good 'soft skills' (MacLeod, 2000). Found conceptually in the affective domain as the external expression of internalized emotion through attitudes and values (Anderson & Krathwohl, 2001; Krathwohl, Bloom & Masia, 1964; Gagne, Briggs & Wagner, 1992), workplace training in these content areas is being implemented to remedy this skills gap.
At the same time this skills gap is occurring, a technological revolution in education (Peters, 2002) has begun. How then, can we integrate technology and remedy the lack of attention to affective learning outcomes at the same time? Previous studies suggest that teaching and learning with technology has a small, positive, non-significant (p > .05) effect on students' affective outcomes when compared to traditional instruction (Waxman, Lin & Michko, 2003). The goal of this research is to verify whether synchronous and/or asynchronous online technologies can produce affective learning outcomes and, if so, to ascertain which mode of online study is most effective.
Background Information
In less formal learning settings, affective outcomes like attitudes, values, motivation, discipline, thinking and communication skills are effectively learned through a process called socialization. The socialization process occurs when agents of socialization (parents, teachers, peers, significant others, mass media) provide models, examples, suggestions and context for appropriate social development (Shaffir & Turowetz, 1983). Evidence of socially appropriate attitudes, values, etc. is observed and measured in current curriculum, but not taught. Instead, affective outcomes are expected to occur through informal learning, or the school based socialization process. This type of development is supported through social interaction, defined as mutual or reciprocal action with 'others' in a social environment.
What mechanisms, then, will foster affective outcomes in education that occur in virtual learning spaces, or online, where there is less opportunity for informal engagement? There is limited research on the use of technology to generate affective outcomes using online methods. There is ongoing debate whether the use of the technology improves learning or whether it is the design of the instruction that improves learning (Clark, 2001; Kozma, 2001). Previous studies suggest that teaching and learning with technology has a small, positive, non-significant (p > .05) effect on students' affective outcomes when compared to traditional instruction. (Waxman, Lin & Michko, 2003). Meta-analysis studies on media research showed that there are significant learning benefits when learning from media (Clark, 1983); however, the reason given for the learning benefits is not because of the media, but rather because of the learning strategies employed when media is used..
Learning strategies that support interpersonal dialogue are paramount to generating affective outcomes or soft skills. The most notable distinction between types of dialogue in online environments is that of synchronous (at the same time, in real time) vs. asynchronous (separately, at different times) engagement with other participants in online learning activity. In addition to multiple opportunities for online interaction, there are many benefits of teaching soft skills using online methods.
Benefits relate first to accessibility and timing. Online learning environments allow learners to learn in the context of their individual workplace, interactively, and allow for collaboration using the technology (Ring & Mathieux, 2002). The opportunity to contextualize, and practice in-situ, is of great benefit. Learners can be located anywhere and complete their training as long as they have the technology to access the learning materials. As learners can access the learning materials anytime, they select the time they learn best to complete their coursework. This helps increase the satisfaction and completion rates. From an instructional point of view, material in electronic format is easy to update. Students can receive updates and feedback in a timely fashion. In addition, the changing nature of some occupations requires just-in-time training in order for workers to maintain currency and competence.
A major benefit of using online methods in the development of soft skills is consistency of training within organizations. This is true for organizations that have offices in multiple geographic locations. Workers access the same training materials while learning from a central location. This is done while allowing the necessary customization, such that doesn't compromise consistent outcomes. Instructional designers of online training materials can take advantage of the technology to customize learning to meet the learning style of individual workers in individual locations.
Beyond technology, timing and access, so too is benefit found in the opportunity for ongoing dialogue; both online learning and affective outcomes are fostered by interaction. The socialization process that supports affective outcomes face-to-face can be replicated online, shaped in online delivery by what Moore (1993) calls 'transactional distance.' Transactional distance emerges in the relationship between type of dialogue and the amount and nature of direction or structure. The combination of these factors yields a level of transactional distance or level of intimacy; high structure and low dialogue yield remote transactional distance and low structure and high dialogue yield close transactional distance. The juggling of structure and dialogue to manage transactional distance results in varying levels of learner autonomy and, in turn, improves course completion. However, the impact on learning outcomes of close versus remote transactional distance is unclear; given the nature of affective learning, we propose that close transactional distance is required to create a socialization affect.
Transactional distance refers to variation in amounts and types of dialogue, and amounts and types of structure. This is also the variation between synchronous and asynchronous interaction. Synchronous interaction of any kind requires high engagement with faster processing and response time. Here, interaction between learner and instructor occurs in real time using two-way text, audio, and/or video. Conversely, "the use of asynchronous learning requires planning, structure" (Garrison, 2003, p. 10), and therefore more engagement, time and thought. The learning activities of engagement, reflection and time invested all lead to increased learning outcomes, suggesting that asynchronous environments is more likely to foster more salient affective learning outcomes.
In asynchronous interaction, there is a delay in communication between the instructor and learner. Does this delay, for example, improve or hinder affective learning outcomes? Modified from the work of Goodyear (2001), the strengths and weaknesses outlined in Figure 1 identify real differences between synchronous and a synchronous learning.
Figure 1. Differences in synchronous vs. asynchronous delivery modes
| Synchronous | Asynchronous | Strengths | Weaknesses | |
| Text-based | e.g. IRC (Internet relay chat), MOOs and MUDs | e.g. email, CMC | encourages clarity of expression, formalisation of knowledge etc.; indexable, searchable; small data files | hard to capture real world working practices or tacit knowledge |
| Multi-media | e.g. live video-conference; shared workspace | e.g. video-on-demand; video-mail | vivid; rich; allows 'showing' as well as 'telling'; can be quick to produce and 'read' | time-consuming to produce; hard to index & search; large data files |
| Strengths | supports interactive communication; timely; sense of event & audience | time to reflect; flexible use of time | ||
| Weaknesses | inflexible use of time; may not scale up to large numbers. May not be suitable for students in different time zones. | Interaction can be slow or cumbersome |
Based on these delivery differences, it is reasonable to assume there will be differences in affective outcomes across platforms. This research will test whether differences in structure and dialogue, as represented in synchronous vs. asynchronous delivery methods, will yield differences in close transactional distance, and thereby impact online learning outcomes in the affective domain.
In summary, a rather lacidasical approach to fostering affective outcomes in public education has contributed to a major skills shortage in the workplace. Clark (2005), studying this shortage in the UK, identified little evidence of the nature and effectiveness of online education in the generation of affective outcomes. Human Resource and Skill Development Canada observed that employees in call centre/help desk environments must be able to handle not only the technological aspects of their job, but a variety of customer queries in an appropriate manner, using good soft skills (MacLeod, 2000). These soft skills, defined as self-awareness, analytical thinking, leadership skills, team-building skills, flexibility, the ability to communicate effectively, creativity, problem-solving skills, listening skills, diplomacy and change-readiness, are lacking in the workplace.
Methods
The distinction between audio and text-based online interaction is the central point of comparison for this study. The indelible characteristic of the written word adds an intensity and thought pattern different from audio conversation. It is suggested that this difference adds to reflection, amount of cognition (Garrison, Anderson & Archer, 2000), and, in turn, the embedding of new learning (Wahlberg, 1988). "Written communication in an online context provides for a systematic approach to constructing meaning, particularly in difficult and ill-defined content areas" (Garrison, 2003, p. 10). Learning in the affective domain may benefit from the reflection and time invested in text-based communication environments.
The study employed a two-group, quasi-experimental design, with variation of the treatment variable, type of online interaction. Two platforms were chosen, one based on synchronous communication (ElluminateLive) and the other Web-based delivery with asynchronous conferencing, after peer reviewed discussion and validation of characteristics in the platforms. A course in customer service, with required affective outcomes, acted as the education intervention. Participant outcomes were measured with a pre/post self-administered soft-skill self evaluation (see Appendix A) and a content-based examination.
Estimates of soft skills competency was evaluated in a self-test before training, to establish an entering baseline. This self-assessment, also in the pilot stage, was completed the week prior to training via an e-mail request. Nine soft skill concepts, as identified by MacLeod (2000) were each given two items. Each item offers a statement regarding participant behavior. Participants responded about the frequency of his/her own behavior on a Likert-type scale.
The treatment variable, or course, was designed with reference to the generation of affective outcomes for online environments. Four modules, with three topics in each module, made up the structural shell of the course. Each topic followed the same instructional pattern: introduction, objectives, personal objectives, content presentation, demonstration, application, and personal reflection.
Facilitation support for affective outcomes was offered by offering the following learning events. A high touch learning environment was created in both WebCT and ElluminateLive. High touch learning environments include facilitation that is learner-centered, with demonstrable validation and, where possible, accommodation of student needs and objectives. Baker (2003) identified that "instructor immediacy [i.e., teaching presence] was more predictive of affective and cognitive learning" than "whether students felt close to each other" (p. 1); facilitator immediacy in reference to requests and feedback was a critical facilitation goal. This was accomplished online through timely response to postings, e-mails, telephone calls and questions asked in synchronous discussions. Facilitator identification of standards, requirements, customer service models and exemplary customer service models provided external reference for individual actions. Self-awareness exercises, reflection opportunities, practice requirements and application exercises were embedded in the instructional design.
At course completion, a final exam was provided via email for ElluminateLive participants. WebCT participants took the final exam online. The final exam was made up of multiple choice and short answer questions regarding appropriate responses, possible courses of action, and attitudes toward customers. Desired soft skills were embedded in correct answers on the final exam questions.
Finally, data was collected from students in the form of a course evaluation. The course evaluation questions were housed on Zoomerang.TM Participants were sent the web page address for the evaluation site just after completion of the final exam.
Findings
Two notable qualitative differences were identified by the facilitators across treatment groups. ElluminateLive attendance was highly variable and access to content, provided to students in a digital repository, was technologically challenging and hindered access to course material for the first week. However, engagement in weekly classes was high among those who participated and enthusiasm for the learning platform was notable. The WebCT group members were heavily involved in the course and used the multiple learning opportunities on the site: content support; text, audio, video material; asynchronous threaded discussion; synchronous chat; group work and learner presentations – all available around the clock, seven days a week.
Three data points were used to assess differences across the two groups in relationship to skill and knowledge regarding soft skills in the workplace. First, the difference between means on the soft skill assessment tool (Cleveland-Innes & Ally, 2004) on the pre-test at time one (beginning of course) and the post-test at time two (end of course) were measured for each group. Reliability appraisal for the instrument yielded an acceptable Cronbach's Alpha of .7011. The mean score allowed us to determine which groups realized more change in soft skill assessment, by comparing the mean scores across groups.
In the self-assessment pre-test, there was no significant difference between entering soft-skill competency. This is the case for each individual item in the test, and aggregate data for each group. In this situation, no significant difference is the desired situation. Participants in both groups entered the experiment with roughly equivalent competencies.
However, no significant differences were found between treatment groups in the self-assessment scores from time one to time two. The difference between pre and post test means were very similar (WebCT, n=56, 4.25; ElluminateLive, n=45, 3.9) yielding a very small, statistically insignificant difference across groups. This suggests that the two platforms were equally effective in the increase in soft skill competency.
Final exam scores, however, did indicate significant differences across groups. Results of final exam scores are reported in Table 1 below.
Table 1. Final exam scores
Group |
N |
Exam Mean |
Standard Deviation |
WebCT |
56 |
87.6 |
10.42 |
ElluminateLive |
45 |
79.3 |
13.91 |
Significance was set at p = .10 as acceptable for use with this pilot data. The difference between group means has a numerical score of 8.3 and a test of significance at p = .069. Variation among exam scores was greater among the ElluminateLive group.
The third set of data was collected through the course evaluation. Response rate for the evaluation was 96%. Differences in the response to the online course, offering the same content and learning exercises, were notable on some questions. Table 2 outlines mean responses to evaluation questions across treatment groups.
Table 2. Evaluation question responses
Question |
WebCT |
ElluminateLive |
I am satisfied with what I learned |
25% |
54% |
The content of course was relevant to work |
40% |
72% |
The content was interesting to me |
26% |
73% |
This course met my expectations |
15% |
36% |
Would recommend this course to others |
50% |
73% |
Discussion
Results of the pilot studies suggest that, under the conditions of asynchronous, text-based interaction participants realize greater affective learning gains than those engaged in synchronous, primarily audio interaction (ElluminateLive). However, these gains were identified in an exam measuring cognitive awareness and knowledge about the appropriate use of affect in customer service situations. This is not a measure of attitudinal change; attitudinal change did occur but gains were not substantively or statistically significant across platforms.
Of particular interest is the disparity in perception of participants taking the same program, with the same content and learning exercises, delivered in different learning management structures. Less than half (40%) WebCT students felt the content was relevant to work, but 72% of ElluminateLive students did. The pattern of high agreement for those in ElluminateLive was consistent on all questions. Notable is the pattern among the responses; half the WebCT students said they would recommend the course, but only 15% felt the course met expectations.
It seems reasonable to suggest that the ElluminateLive participants were generally more satisfied with the course experience, indicating in greater numbers that they would recommend the course to other (73%), the content was relevant (72%). This finding suggests that synchronous interaction results in a higher level of personal satisfaction when compared to the asynchronous interaction mode. One possible reason is because in the synchronous mode, learners are able to ask questions and receive immediate feedback. At the same time, if learners want to delve more into the conversation, there can do so easily using the synchronous mode.
The concept of transactional distance may also help explain why this experience was different across platforms. Both groups engaged with the same content, completing the same assignments and learning exercises. The difference between groups had to do with access to the written material for the course. WebCT students had material available online, while ElluminateLive students had to access course guides through a separate site. Differences in type of interaction led to closer transactional distance (low structure, high dialogue) in ElluminateLive and more remote transactions (high structure, low dialogue) in WebCT. It is reasonable that facilitation characterized by close transactional distance would foster achievement of affective outcomes. We believe these facilitation strategies include the presence of emotional and psychological safety, opportunity for interaction, the observation of demonstration of new and appropriate models of behavior, activities that promote self-awareness, activities that promote self-reflection and the opportunity for application of new behavior.
Conclusions
Results from this initial study of online learning and affective outcomes are inconclusive in relation to the comparison of delivery outcomes, but hopeful in terms of working toward soft skills and affective gains using online delivery. Both groups performed well on an exam testing knowledge of appropriate affect in customer service environments. The soft skill assessment showed some gain from time one to time two in both platforms.
Most interesting is the affect shown by participants about the course itself. Those in the ElluminateLive group demonstrated a more positive response to the course than those in WebCT. Participants were randomly assigned to different platforms, were in the same occupation and working for the same or similar organizations. There are no systematic differences we can identify between groups that may account for the different response to the course. We suggest that transactional distance (Moore, 1993) in ElluminateLive was diminished and may have resulted in a more engaging, positive experience for participants.
There may have been differences in the sense of community felt across groups, with the ElluminateLive group feeling more connected through the challenges they experienced together. There is a relationship between social presence, cognition and affective outcomes in the online environment. According to Garrison, Anderson and Archer (2000) social presence is defined as "the ability of participants in the community of inquiry to project their personal characteristics into the community, thereby presenting themselves to other participants as 'real people.'" The "primary importance" of social presence "is its function as a support for cognitive presence." Most importantly for this discussion "when there are affective goals for the educational process…then social presence is a direct contributor to the success of the educational experience." (p. 4).
Future research
This study will be replicated to continue assessment of affective outcomes in online learning environments. Adjustments will be made to shore up any differences that may affect outcomes. The facilitator will be consistent across platforms. Access to material for ElluminateLive participants will be improved and self-test reviews will be created at the end of each module. Asynchronous interaction in WebCT will be measured to evaluate the amount of time participants engaged with others in the course; facilitation will work to equalize the amount of interaction with the hours spent interacting through ElluminateLive.
Online delivery in ElluminateLive and WebCT have differences that impact delivery beyond the differences of synchronous vs. asynchronous communication. Synchronous interaction in ElluminateLive is primarily audio, not text-based. Audio interaction does not require the same level of reflection or thoughtful message construction that text-based communication does. This may have significant impact on learning outcomes (Wagner, 1998). Secondly, WebCT offers flexible interaction, giving students the opportunity to interact for as long as they like, whenever they like. In the facilitation of affective learning outcomes, this may critical. Further research needs to test this, and other, key premises about the generation of affective learning outcomes in online learning environments. More research is also needed to determine whether a blended approach, using both synchronous and asynchronous interaction methods, is more effect than individual approaches.
According to Southwell (2000), education incorporating appropriate curricula in the areas of personal development, values education, interpersonal relationship skills, environmental, and peace studies are keys to ensuring the development of globalization and internationalization required in the 21st century. This study and others like it will ensure that, as online learning is adopted, appropriate design and delivery for all types of learning outcomes will be identified.
Acknowledgment
We thankfully acknowledge Athabasca University's Academic Research Committee for funding this research.
Contact Details
Dr. Martha Cleveland-Innes Phone: (780) 675-6426 Fax : (403) 238-7762
Dr. Mohamed Ally Phone: (780)675-6406 Fax: (780)675-6170
References
[1] Anderson, L.W. & Krathwohl, D. (Eds). (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's taxonomy of educational objectives.New York: Addison Wesley Longman.
[2] Baker, J. D. (2002). An investigation of relationships among instructor immediacy and affective and cognitive learning in the online classroom. The Internet and Higher Education, 7(1), 1-13.
[3] Clark, D. (2005) Softskills and E-learning. London: Epic Performance Improvement Limited.
[4] Clark, R.E. (1983). Reconsidering research on learning from media. Review of Educational Research. 53(4), 445-459.
[5] Clark, R.E. (2001). A summary of disagreements with the "mere vehicles" argument. In R.E. Clark (Ed.), Learning from media: Arguments, analysis, and evidence(pp. 125-136). Greenwich, Connecticut: Information Age Publishing Inc.
[6] Claxton, G. (1989). Cognition doesn't matter if you're scared, depressed or bored. In P. Adey (Ed), Adolescent development and school science (pp. 155-161). Lewes, UK: Falmer Press.
[7] Cleveland-Innes, M. & Ally, M. (2004). Affective learning outcomes in workplace training: A test of synchronous vs. asynchronous online learning environments. Canadian Journal of University Continuing Education (CJUCE) Journal, 30(1), 15-35.
[8] Cole, R.A. (2000). Issues in web-based pedagogy: A critical primer. Westport, CT: Greenwood Press.
[9] Gagne, R. M., Briggs, J.J. & Wagner, W.W. (1992). Principles of instructional design. Fort Worth, TX.: Harcourt Brace Jovanovich College Publishers.
[10] Garrison, D. R. (2003). Cognitive presence for effective asynchronous online learning: The role of reflective inquiry, self-direction, and metacognition. In J. Bourne & J. C. Moore (Eds.) Elements of quality online education: Practice and direction. (p. 47-58). Needham, MA.: Sloan-C.
[11] Garrison, D. R., Anderson, T. & Archer, W. (2000). Critical Inquiry in a text-based environment: computer conferencing in higher education. Internet and Higher Education, 2 (2), 87-105.
[12] Goodyear, P. (Ed.). (2001). Effective networked learning in higher education: notes and guidelines. Retrieved September 1, 2007 from http://72.14.203.104/search?q=cache:brgdlcmiFKQJ:csalt.lancs.ac.uk/jisc/guidelines_final.doc+"synchronous+vs.+asynchronous"+online+"TRAnsactional+distance"&hl=en&gl=ca&ct=clnk&cd=5&lr=lang_en
[13] Gronlund, N.E. (2000). How to write and use instructional objectives. Toronto: Prentice-Hall.
[14] Kozma, R.B. (2001). Counterpoint theory of "learning with media". In R.E. Clark (Ed.), Learning from media: Arguments, analysis, and evidence (pp. 125-136). Greenwich, Connecticut: Information Age Publishing Inc.
[15] Krathwohl , D.R., Bloom, B.S., & Masia, B.B. (1964). Taxonomy of educational objectives: Handbook II: Affective domain. New York: David McKay Co.
[16] Leonard, C., Bourke, S., & Schofield, N. (2004). Affecting the affective: Affective outcomes in the context of school effectiveness, school improvement and quality schools. Issues in Educational Research, 14(1), 1-28.
[17] MacLeod, A. (2000). The importance of soft skills in the current Canadian labour market. Sectoral and Occupational Studies Division of Human Resources Development Canada, April.
[18] Moore, M. G. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22-38). New York: Routledge.
[19] Peters, O. (2002). Distance education in transition: New trends and challenges. Oldenberg, Germany: Bibliotheks- und Informations sytem der Universitat Oldenburg.
[20] Postman, N. (1997) The end of education: Redefining the value of school. New York: Alfred Knopf.
[21] Ring, G. & Mathieux, G. (2002). The key components of quality learning. Paper presented at the ASTD Techknowledge 2002 Conference, Las Vegas, Nevada, USA.
[22] Shaffir, W.B. & Turowetz, A. (1983). Socialization. In M.M. Rosenberg, W.B. Shaffir, A. Turowetz, & M. Weinfeld (Eds.), Sociology. Toronto: Methuen.
[23] Southwell, B. (2000). The international context of education. In S. Dinham, & C. Scott (Eds.), Teaching in context, pp.123-134. Camberwell: ACER.
[24] Wagner, E.D. (1998). Interaction strategies for online training designs. Proceedings of the 14th Annual Conference on Distance Teaching & Learning, Madison, Wisconsin, USA.
[25] Walberg, H. (1988). Synthesis of research on time and learning. Educational Leadership 45(6), 76-85.
[26] Waxman, H. C., Lin, M.-F., & Michko, G. M. (2003). A meta-analysis of the effectiveness of teaching and learning with technology on student outcomes. Retrieved 29 May, 2006, from http://www.ncrel.org/tech/effects2/
Appendix A
Always |
Often |
Occasionally |
Seldom |
Never |
||
1. |
I use a model of problem resolution /trouble-shooting when reviewing caller information. |
5 |
4 |
3 |
2 |
1 |
2. |
I seek to understand customer expectations through careful listening. |
5 |
4 |
3 |
2 |
1 |
3. |
I reflect on my behavior as I proceed through the call. |
5 |
4 |
3 |
2 |
1 |
4. |
I look for multiple responses to caller requests. |
5 |
4 |
3 |
2 |
1 |
5. |
I adjust my service methods according to customer needs. |
5 |
4 |
3 |
2 |
1 |
6. |
I demonstrate frustration at times during the call process. |
1 |
2 |
3 |
4 |
5 |
7. |
I uniformly follow the same call process for each call. |
1 |
2 |
3 |
4 |
5 |
8. |
I recognize my strengths and weaknesses in serving customers. |
5 |
4 |
3 |
2 |
1 |
9. |
I thoroughly review information presented to me. |
5 |
4 |
3 |
2 |
1 |
10. |
I never have the opportunity to change company procedure/policy to better support customers. |
1 |
2 |
3 |
4 |
5 |
11. |
I clarify that callers have understood what I have said. |
5 |
4 |
3 |
2 |
1 |
12. |
I represent my company in positive terms. |
5 |
4 |
3 |
2 |
1 |
13. |
I understand the problem solving process. |
5 |
4 |
3 |
2 |
1 |
14. |
I divide complex problems into workable, understandable units. |
5 |
4 |
3 |
2 |
1 |
15. |
I take problems that can be solved organizationally to the appropriate people. |
5 |
4 |
3 |
2 |
1 |
16. |
I present my ideas in a way that is understandable to others. |
5 |
4 |
3 |
2 |
1 |
17. |
I see callers' concerns as an opportunity to problem solve. |
5 |
4 |
3 |
2 |
1 |
18. |
I put myself in the 'shoes of the other' when listening to a caller's concern. |
5 |
4 |
3 |
2 |
1 |
19. |
I identify cause and effect relationships in my approach to problems. |
5 |
4 |
3 |
2 |
1 |
20. |
I consider my audience when crafting a message to another person. |
5 |
4 |
3 |
2 |
1 |
21. |
I take initiative to resolve complex problems, even when they may be outside my job description. |
5 |
4 |
3 |
2 |
1 |
22. |
I have no need to verify my understanding when listening. |
1 |
2 |
3 |
4 |
5 |
23. |
I brainstorm possible solutions with new ideas and insight. |
5 |
4 |
3 |
2 |
1 |
24. |
I speak with respect about others when conversing with customers. |
5 |
4 |
3 |
2 |
1 |
25. |
I accept changes in policy/ procedure as a normal part of the work experience. |
5 |
4 |
3 |
2 |
1 |
26. |
I take note of others' responses to my behavior. |
5 |
4 |
3 |
2 |
1 |
27. |
I adjust to required changes in my work life. |
5 |
4 |
3 |
2 |
1 |