About EURODL |  Sections |  Subscription |  For Authors |  Special Issues |  Archives   Peer-reviewed articles |  Brief items |  Reviews

NEW

Best of EDEN 2018 Special issue
 

There are not any recent contributions.

There are not any recent contributions.

Archives

2020/I 2019/II 2019/I 2018/II 2018/I 2017/II 2017/I 2016/II 2016/I 2015/II

2015/I 2014/II 2014/I 2013/II 2013/I 2012/II 2012/I 2011/II 2011/I 2010/II

2010/I 2009/II 2009/I 2008/II 2008/I 2007/II 2007/I 2006/II 2006/I 2005/II

2005/I 2004/II 2004/I 2002 2001 2000 1999 1998 1997

EURODL Mailinglist

2851 subcribers
 

EURODL Visitors

back

Impact of Learning Modalities on Academic Success

Fabien Fenouillet & Jonathan Kaplan [kaplan@kaplan-consultants.org]
Education and Training Research Centre [CREF – EA 1589]
West Paris Nanterre – La Défense University [UPX]
200, avenue de la République
F–92001 Nanterre, France
Phone: +33 970 448 282

Abstracts

English

This study is based on the analysis of academic results of 692 undergraduate and graduate students in two disciplines in a French university who attended their courses using one out of four possible learning modalities. Within the two disciplines, Art History and Educational Sciences, students chose between face-to-face learning (on campus), paper-based distance learning (correspondence), Web-based distance learning (e-learning) or a blend of Web-based and face-to-face learning (combined) modalities. Comparison, using two indicators of overall academic success between the four learning modality types was conducted. The analyses revealed that e-learners attending at least one course on-campus (combined modality) did just as well as students attending all their courses using the on-campus only modality. Furthermore, depending on the discipline, e-learners' academic success was higher than correspondence learners', albeit not as high as on-campus and combined learners' achievements.

French

Cette étude porte sur l'analyse de la réussite universitaire des étudiants de licence et de master première année en fonction de différentes modalités de formation (université française) et de deux disciplines (histoire de l'art et sciences de l'éducation). Quatre modalités de formation ont été prises en compte : en présence, à distance par correspondance postale, à distance via une plateforme Internet et en présence augmentée par l'utilisation d'une plateforme Internet (LMS). Les analyses prenaient en compte le dispositif de formation dans sa globalité ce qui permettait d'estimer l'impact d'un mode de suivi mixant des unités en présence et à distance. Les résultats permettent de conclure que les étudiants totalement à distance réussissent moins bien que ceux en présence. Les étudiants en présence qui suivent certaines de leurs unités totalement en e-learning ont des résultats équivalents à ceux qui suivent toutes leurs unités en présence. Par contre, en fonction de la discipline, le e-learning permet d'atténuer l'impact négatif sur la réussite aux examens dans le cadre d'un enseignement totalement à distance.

Keywords

e-learning, distance learning, blended learning, academic success, art history, educational sciences.

Introduction

Information and communication technologies (ICTs) have penetrated increasingly into university courses since the 1990s in France, as elsewhere. Reports from the USA have shown an increasing share of Internet use in teaching, mainly in distance education (Wirt et al. 2004; National Center for Education Statistics 2002). Nevertheless, if the technology potentially can facilitate diverse forms of teaching, several researchers have questioned the pertinence of such novelty (Fenouillet and Déro 2006; Albero 2004; Phipps and Merisotis 1999).

Although distance education facilitates access to university courses for specific categories of the population (employees, geographically removed students, to give just a couple of examples) distance education has been considered since its beginning as an alternative to face-to-face learning and not as a learning modality in its own right. Numerous studies have pointed to the limits of this substitute to face-to-face forms of learning. It occurs for example, that the dropout rate of distance learners is much higher in comparison to students in face-to-face settings. (Carr 2000; Diaz 2000; Easterday 1997; Roblyer 1999). A study that was conducted by Vergidis and Panagioutakopoulos (2002) at the Hellenic Open University in Greece shows that several factors are related to this, such as professional activity or a miscalculation by students of time they can devote to their learning. Indeed, people engaged in distance learning do not have the same constraints as those studying in face-to-face settings. Nevertheless, Pascarella and Terenzini (1980) have shown that students that have frequent contact with the university in which they are enrolled (and in particular informal contact), present a lower dropout rate. Internet, with the communication means that are part of it (e-mail, instant messaging, forums etc.), appears to introduce non-negligible progress in respect to distance education that takes place solely using postal services.

Enthusiasm is often connected to the advent of newer practices, mainly those emerging from asynchronous services such as e-mail, forums, file exchange. Asynchronous communication enables students to turn to resources at their own pace and above all to take the needed time for reflection, when they reply to an e-mail or to a forum post, for example. For Spencer and Hiltz (2001), asynchronous learning is superior to other forms of purely synchronous learning on several criteria. This conclusion can be considered in the light of conclusions arrived at in numerous other studies in the narrower framework of computer-based instruction, or to those several hundred studies that have been the object of meta-analyses (Cavanaugh 2001; Cavanaugh et al. 2004; Clark 1985; Liao and Bright 1991; Lipsey and Wilson 1993; Waxman et al. 2003). These studies point to positive effects that different forms of technology have on learning. Nonetheless, they are primarily concerned with computer-based instruction and not with the use of Internet for learning. Moreover, even at this level, authors are far from being unanimous as some find the effect of technology as such to be nil (i.e. the media has no impact) and when effect is found, it is attributed to pedagogical methods embedded in the tools  (Clark 1983, 1985, 1994; Gagne et al. 1992; Joy and Garcia 2000). Fenouillet and Déro (2006) use different criteria enabling a finer grained conclusion through results of 34 studies comparing face-to-face with online courses. Criteria used were exam grades, knowledge level tests, student satisfaction and dropout rates.

Online and face-to-face courses generally appear equivalent when comparing exam scores. At odds, learner satisfaction tends to be higher in face-to-face settings. As to student dropout, a small number of studies points to a higher dropout rate in face-to-face settings.

Nevertheless, as Phipps and Merisotis (1999) have noted, comparisons that attempt to clearly reveal the superiority of online learning over face-to-face learning pose countless methodological pit-holes. One of the most notable of which is that the use of Internet enables to introduce blended learning modes that combine both face-to-face and distance learning. Haeuw et al. (2001) suggest a continuum of technology use at which at one end technology is present in face-to-face learning whilst at the other it is used entirely at a distance. In effect, ICT in education can be used in different proportions between these extremes making it difficult to distinguish between educational designs. Haeuw et al. (2001) introduce no less that three intermediary learning forms that can be considered as blended. In this sense blending is considered within the same course. At the same time one can point to the fact that blending can take place within curricula in general. One course can be taught face-to-face while another may be delivered at a distance over Internet. The combination between both modes presents the advantage of implicating students in campus life even when they are following most of their studies at a distance. Considering this, it is possible to hypothesize that mixing these in the program can enable breaking student isolation when learning at a distance, to a greater deal than ICT tools would, as have noted Pascarella and Terenzini (1980) in the framework of partially-based distance learning using postal services.

The terminological abundance that is presently in use to designate online learning translates the relatively new difficulty in characterizing the nature of different designs. The same educational design may be considered as open and distance learning (ODL), as e-training, e-learning, blended learning, self-education or even e-teaching. Beyond terminological nuances we will be using the term e-learning. In our study, which this paper presents, two seemingly essential aspects impact learning and academic success. The first is distinguishing learning that takes place in part from learning that takes place entirely at a distance. The second is the type of media being used to access information. It may either be fully electronic, using ICTs, or be paper-based using postal services which, in our case, was not intended to enable interaction with the instructor or with other students.

Added to the complexity inherent in the multiplication of possibilities that technologies offer, there is another no less complex dimension to consider which is the discipline. In teaching mathematics, Smith and Ferguson (2005) show that student dropout rate is significantly higher when e-learning is used on its own, in comparison to other disciplines taught that way. Whereas dropout rates in face-to-face designs are similar between disciplines (insignificant differences). For the authors (Smith and Ferguson 2005), the discrepancy is explained by the inadequacy of tools used in e-learning for teaching mathematics. Smith et al. (2008) in another study suggest that the problem of using Internet-based tools in respect to different disciplines is much larger and does not pertain only to mathematics. Smith et al. (2008) base their work on Biglan's (1973) classification, arrived at after having addressed academic teaching staff in 30 disciplines who were asked to order disciplines by domain proximity. This classification made it possible to establish groups of fields of study that are organized on different dimensions. The most known grouping is no doubt the one based on hard sciences (physics, chemistry, mathematics, botany etc.) and soft sciences (philosophy, sociology, psychology etc.). Another grouping enables distinguishing between theoretical sciences (for example; physics and mathematics for hard sciences, or history and philosophy for soft sciences) from applied sciences (for example; mechanical engineering and medicine for hard sciences, or educational sciences and finance for soft sciences). It appears that values, objectives, perspectives that are underpinning these dimensions are not the same which leads to notably different use of services implemented in electronic platforms or learning (content) management systems [LMS or LCMS] that are used in online courses. Neumann (2001) explains that learning soft sciences relies on more reading time, seminars and tutoring whereas learning hard sciences requires hands-on (laboratory) activities with preference for exercises and error analysis. Hard sciences have hermetically structured curricula around concepts and principles highly linked one to the other while soft sciences are more open and have a much more approximate organization. Instruction in hard sciences attaches more importance to learning facts, principles and concepts whereas soft sciences summon more general knowledge, reflection, critical thinking and creativity.

White and Liccardi (2006) put forward the idea of "technological affordance" that may be a function of the activity or process supported or encouraged by a particular technology. For example, technologies that favor communication and discursive reasoning (such as forums) can facilitate critical thinking. Several studies (Jones et al. 2004; Smith et al. 2008) point to diverging use of digital resources depending on the discipline and subject-matter. This affects progression of digital resources usage and networked learning in universities.

Study basis, limits and hypotheses

The study relies on grades of students in programs from two disciplines in which partaking in courses was either through e-learning, postal mail or the more traditional on-campus face-to-face mode. Both domains can be classified as soft sciences (Biglan's classification 1973). Whilst one of them represents applied sciences (Educational Sciences), the other represents theoretical sciences (Art History).

The end of year results are indicators reflecting academic success. The data are relatively complex to analyze as there are at least two types of results: getting credit for the academic year (passing) and the average grade obtained by each student across all courses. In order for both to be accounted for, it is required that the student completes all courses of both the year's semesters. The fact of not completing all courses may be considered as failure by some of the students, while for others this may be part of a more global strategy. For example, students who are professionally active in parallel to their studies, who are present in great numbers in distance learning courses, may use a strategy of getting credit for courses over several years. Such students may decide to get credit for only a few courses during the academic year but may end up with excellent results once all courses are completed over several years. Put differently, the fact of not completing all courses in one year could be considered as a strategy for success and not failure, specifically when the strategy is freely chosen by the student. Therefore, it is important to consider both results in order to see if the analyses point to identical conclusions. In addition, assessment in each course may vary. Failure grades may be attributed to students that have not handed in mid-semester work in an ongoing assessment design. In other cases, credit for a course that relies on handing in a final assignment or paper may, if the student does not hand it in, lead to course failure or to the course being designated as incomplete. The grade compensation system also makes it possible for a student to fulfil requirement for a course without investing in passing it, as other courses will compensate and still make it possible for him or her to pass the academic year. Here again, considering these dealings with the university accreditation system, it seems worth taking into account students who have failed but who may have played up to the system.

All this indistinctness in the university's assessment system, briefly presented here, makes one aware of the inherent limits of the indicators used. University assessment does not truly reflect ability, knowledge or students' capacities, but a means for measurement subject to numerous constraints, as any measuring apparatus may be subject to.

Albeit the numerous limits, the assessment set-up should enable testing some hypotheses. Overall, one may expect that academic success in courses taken at a distance will be lower than that of students attending courses on campus. However, the weight of any difference must be adjusted by taking two factors into account. The first is the presence of blended learning that can take on two forms. Form A is the combination of courses, some taken on campus while others being attended wholly online. As Pascarella and Terenzini (1980) have demonstrated, this should favor student academic success. Form B where blending takes place within a course, in which course design involves both distance learning and on-campus sessions. Our hypothesis is that form B of blending does not result in significant difference compared with face-to-face only designs as it does not generate student isolation which we might consider as adverse.

The second factor is the use of ICTs. In general, means made available by these should enable to better the success of distance learners. Nevertheless, it is hypothesized that this bettering is related to technological affordance (White and Liccardi 2006). At this level, the interfaces for theoretical domains such as mathematics, require relatively sophisticated underlying technical infrastructures in order to be efficient (Smith and Ferguson 2005). The LCMS that was used in our study^[1] has not been the object of in-depth study to adapt it to different disciplines. However, as the LCMS facilitates communication, it holds greater technical affordance for applied sciences (White and Liccardi 2006; Smith et al. 2008). Consequently, it is hypothesized that academic success of students who use distance learning in Educational Sciences would be higher in comparison to that of students in Art History.

Finally, it should be noted that both disciplines are not structured the same way. Art History has a relatively traditional curricular structure with three years leading to graduation and two extra to a Master's degree. Educational Sciences on the other hand, start at third year at undergraduate level (the first two being completed in other disciplines); and similarly, two extra years to reach a Master's degree. Also, student results taken into account only include the first year at master's level as the second year of post-graduate studies in French universities is subject to numerus clausus. All these considerations hence make comparison difficult as some correlations are impossible because of discipline specifics.

Methodology

The population is made up of 2847 students enrolled at a university in greater Lille during the 2004-05 academic year. Students enrolled were first, second and third year undergraduates plus first year graduates preparing for a master's degree.

Two disciplines were included in the research, Educational Sciences and Art History. They were chosen because of the various learning modalities that were designed into the programs. Students in both disciplines attended courses in face-to-face settings, at a distance receiving their course material by Post, or used a LCMS. While it is possible to gain access to student grades per course or globally for the academic year, the university databases do not enable retrieving reliable information on learning modalities used in courses. To obtain the information we needed, we opted for using a questionnaire (taking the opportunity of a larger survey conducted by the university on course quality) that we directly addressed to students. We then crosschecked the data collected with data we could access in university databases. Students were addressed the questionnaire at the end of the first semester (end of January and during February 2005 before grades were given) according to their primary learning modality. Students studying in face-to-face settings were addressed during courses. Correspondence distance learners received the questionnaire by Post. E-learners received a personalized e-mail with a link to an online questionnaire. It should be noted that students in Educational Sciences could combine learning modalities. Art History students, on the other hand, could not. They had to choose one singular learning modality out of four available for all their courses. The available modalities are defined hereafter:

• Face-to-face - Students followed all courses on campus with instructors present and without computer use. We shall refer to this modality in this paper using the term on-campus students.

• Paper-based at a distance - Students followed all courses (Art History) or part of their courses (Educational Sciences) at a distance where postal services were used for receiving printed material. We shall refer to this modality in this paper using the term correspondence students.

• E-learning at a distance - Students followed all courses (Art History) or part of their courses (Educational Sciences) at a distance using e-learning. We shall refer to this modality in this paper using the term e-learning students.

• Enhanced face-to-face - Students followed some courses that blended, within the course, face-to-face learning and e-learning. We shall refer to this modality in this paper using the term combined modes students.

We were able to use 692 responses to the questionnaire, representing 24.31% of the total population. 406 were collected using a paper version of the questionnaire and 286 questionnaires were collected using the electronic version. The following analyses were sometimes based on lower than the maximum 692 questionnaires as a result of missing data that led to abandonment of data pertaining to varying numbers of individuals, depending on the analysis.

Table 1. Distribution of Art History learners in respect to the four learning modalities with or without parallel on-campus courses

 

Table 2. Distribution of Educational Sciences learners in respect to the four learning modalities with or without parallel on-campus courses

 

Proportion of professionally active students

Being professionally active is one of the factors that may effect student academic success. It is worthwhile presenting some analysis results, as data pertaining to this was available.

Table 3. Distribution of professionally active learners per discipline

 

Table 3 shows that there are more students in Educational Sciences that are professionally active compared to Art History students (Χ²(2)=22.14; p<.01). Further analysis indicates that the difference in distribution is only true if we compare students studying on campus (Χ²(2)=18.21; p<.01). When these comparisons are done on distance learning students, correspondence (Χ²(2)=2.58; ns) or e-learning (Χ²(2)=4.82; ns), these differences totally dampen. Generally, when taking into account the proportion of passed, failed or incomplete courses students, the non professionally active are more successful than the professionally active students (Χ²(4)=18.19; p<.01). On the other hand, if we proceed with the same analysis while this time taking into account the average of course grades (grading is based on a 0 to 20 mark system), this difference disappears  (F(2, 482)=1.73; ns). The discrepancy between success in terms of passing and success in terms of average grade, points to the limits of these analyses which are based on groups in a natural setting who are subject to multiple variables. It is nevertheless possible to propose an interpretation of these discrepancies. Professionally active students sometimes study for several years using a strategy of getting credit by passing chunks of related courses. Once they have gained credit for all required chunks they get credit for the academic level. This might explain why there are less professionally active students who passed the academic year. On the other hand, once they have ended their studies they attain grades comparable to other students which explains the equivalence in grade averages. Obviously, deeper analysis would be needed in order to test the validity of these two hypotheses, but that is beyond the scope of this presentation.

Results

Three major categories of analysis were conducted as a function of the considered data. The first analyses were done on the entire population in order to examine results of all students in both disciplines. This level enables grasping the differences between academic years of both graduate and postgraduate programs, and between disciplines. Thus, making it possible to establish to which point both disciplines are comparable, plus the degree to which the sample of students that responded to the questionnaire is representative of the population. The two other categories of analysis are discernible from the first, mainly because they pertain to the sample of students that responded to the questionnaire. These two categories are themselves separate as they each pertain to different result types. The first one takes into account students who passed the academic year; whereas the other, students' general average grade. In all analyses, end-of-year results were used (over two semesters) as the French system, even though it has recently adopted a graduate – postgraduate – doctoral studies system (actually called LMD for Licence – Master – Docteur), still functions to a large extent on the principle of annual results.

Analysis on the entire population

Table 4. End-of-year results related to academic level in Art History

 

Table 5. End-of-year results related to academic level in Educational Sciences

 

Grades can be grouped into three categories. Those who passed are those who fulfilled criteria to receive credit for the academic year. Those who failed are those denied credit for the academic year but who nevertheless attempted passing. Lastly those who failed because they did not attempt passing all required courses are designated as incomplete.

Tables 4 and 5 reveal that the proportion of passing students in Art History is different from that in Educational Sciences^[2]. We also note that, depending on the academic year-level, the number of passing students is very different. Educational Sciences students enter studies in the discipline as third year undergraduates and at this academic year-level success in Art History is superior to that of students in Educational Sciences. These data, distributed across the tables in respect to the academic year-level in each discipline, indicate the importance of the two variables, academic year-level^[3] and discipline, in order to analyze student academic success.

Table 6. End-of-year results related to response to the questionnaire

 

There is a significant relationship between the fact of responding to the questionnaire and the end-of-year results (Χ²(2)=94.99; p<.001; cf. Table 6). Table 6 reveals the higher proportion of students who did not complete all courses out of those who did not respond to the questionnaire. Also, students that did respond have a higher success rate than the others. Nevertheless, there is no significant relationship between the fact of having responded to the questionnaire and the discipline (Χ²(1)=.82; ns) or the academic level (Χ²(3)=4.85; ns). The difference between students who responded to the questionnaire and the others may be interpreted as an inability to contact students who had abandoned their studies during the academic year for whatever reason (professional activity, change of program, discouragement etc.). This also suggests that respondents' data presented underestimates dropout rate.

In terms of average year grades (only for those that completed all courses), there is a difference between disciplines F(1,1621)=10.15; p<.01), and between the academic levels in the discipline (F(3,1621)=549.42; p<.001). As Table 7 indicates, means of students' average grade in first and second year present much lower means than third year undergraduate students.

Table 7. End-of-year grade means related to academic level and discipline for all students

 

Differences are mainly related to the first two undergraduate years in Art History. At a comparable level the differences apparent in Table 7 are much smaller. Apart from what can be seen in Table 7, there is a significant difference (F(1,1624)=35.05; p<.001) between those who responded to the questionnaire (11.49) and those who did not (10.21).

Results of students who responded to the questionnaire

Results analyzed in relation to receiving credit for the academic year

There a significant relationship between the learning modality chosen by the student and the end-of-year results (Χ²(6)=92.43; p<.001). This dependency is explained mainly by the low rate of passers among the correspondence learners. Also to be noted is that more than half the students in that modality did not complete all their courses.

Table 8. End-of-year results related to learning modality

 

We note that the correspondence modality presents problematic characteristics. Most students in this modality are in Art History (Table 1 and 2) in which scores are not as good as those of Educational Sciences students (Table 4 and 5). Also, out of the 139 students in this modality (Table 1 and 2) five used parallel on-campus support. Conclusions are very different when the distribution of results is looked at depending on students' discipline of study. In Art History there is a notable relationship between the chosen learning modality and the end of year result (Χ²(4)=72.72; p<.001), while this is not the case in Educational Sciences (Χ²(6)=10.82; ns). In Art History the persistence of this effect is explained by the massive success of face-to-face students in comparison to those learning at a distance (Table 9).

Table 9. End-of-year results related to learning modalities in Art History

 

Table 10. End-of-year results related to learning modalities in Educational Sciences

 

In Educational Sciences we note that correspondence learners succeed less well than those who chose another learning modality (Table 10). However, considering the low number of students who failed (3 out of 382), we pursued with complementary analyses without accounting for them. This time the relationship becomes significant (Χ²(3)=10.23; p<.05) and can be interpreted in respect to students that used a distance learning mode (Table 11). In accordance with our hypothesis, e-learning students did better than correspondence students.

Table 11. End-of-year results, not taking into account failed students, related to learning modalities in Educational Sciences

 

Here too, differences between e-learning students and correspondence students can be explained by other factors. Table 2 reveals that few correspondence learners had parallel on-campus courses, which was not the case for e-learning students. Overall in Educational Sciences, there is a significant relationship between the fact of having taken at least one on-campus course in respect to end-of-year results (Χ²(2)=81.47; p<.001). 74% of students that had taken at least one course on campus passed, while only 44% did when they used distance learning only for all their courses. This result, in accordance with our hypotheses, seems to corroborate the idea that blending learning modes within or among courses in a program has a positive effect on student success.

As we observed earlier (Table 2), students in Educational Sciences combined with their e-learning on-campus face-to-face learning, which was not the case for students in Art History (Table 1). Two of the modalities used in Educational Sciences are of particular interest as in both cases a significant relationship exists with end-of-year results. This is both true for correspondence students (Χ²(1)=4.31; p<.05) and e-learning students (Χ²(1)=4.15; p<.05). Again, we wish to underline that these results do not account for students who failed. In both distance learning modalities, students who had some of their courses take place on the university premises did better. However, proportions are different. Correspondence learners who took on-campus courses all passed (5 out of 5, i.e. 100%; cf. Table 2); whereas, only half of the learners who took all their courses at a distance succeeded (13 out of 26, i.e. 50%; cf. Table 12). 70.1% of e-learning students passed; whereas, 82.9% did when they took at least one course on campus too. It also seems that despite the fact that following at least one course on campus offers better prospects for passing, using e-learning for all of one's courses still renders better results than correspondence students and thus reduces the gap with results from the on-campus modality. A last series of analyses were conducted to pinpoint this hypothesis. In Art History, distance learners' results, whether correspondence or e-learning, do not differ from on-campus ones (Χ²(2)=0.36; ns). In contrast to this in Educational Sciences, though there is no significant difference between on-campus results and distance learners' results in general (Χ²(1)=1.02; ns), a significant difference is noticeable for e-learners in respect to the former (Χ²(1)=3.92; p<.05). Table 12 reveals that this relationship is linked to superior success of students who use e-learning (70% as opposed to 50%). These analyses done on figures pertaining to studies in Education Sciences, seem to indicate that although e-learning does not totally appease all eventually met difficulties resulting from working entirely at a distance, it does substantially limit their effect.

Table 12. End-of-year results, without considering failed students, related to two distance learning modalities in Educational Sciences

 

In respect to students attending their studies on campus, partly or fully, the question remains as to the effect of taking some courses on-line. An analysis of data pertaining to distant students who had followed at least one on-campus course and their distance learning modality, has enabled to establish that there is no significant relation between these two variables (Χ²(3)=4.16; ns). The absence of relationship indicates that on-campus students who received credit for the academic year were not affected by having undertaken some of their studies via e-learning. This effect too, conforms with the hypotheses. The fact of having taken some courses on-line yields similar results as in the case of having attended all courses on the university premises.

Analysis of end-of-year grade means

Overall, a significant difference is noticeable between the various learning modalities (F(3,451)=11.26; p<.001). As Table 13 indicates, correspondence students attained the lowest grades. At odds with this, surprisingly e-learning students have higher grade means than on-campus students. One has to nevertheless be careful when interpreting these results as certain categories of students are over-represented in some modalities. Correspondence students are mainly studying Art History whereas in this discipline students have markedly lower grade means than Educational Sciences students. Complementary analyses taking into account the discipline and the academic level were therefore required in order to attain a higher analysis resolution.

Table 13. Student means related to learning modality

 

A first analysis with the variables, discipline and modality, indicates that only the discipline is significant (F(1,480)=94.02; p<.001)^[4]. The modality is not significant, neither as simple effect nor interacts with the discipline (F(1,480)=.69; ns). A second analysis of variance using the variables, modality and academic level, indicates a significant difference for modality (F(2,446)=4.75; p<.01) as well as for academic level (F(2,446)=60.79; p<.001) but no interaction effect (F(4,446)=1.08; ns)^[5]. The contradiction between these two sets is related to different groupings and to the fact of not accounting for certain categories of data. The differences in student numbers between the various academic levels, modalities and disciplines, undermine the possible interpretations. A further study at the level of academic disciplines should enable to bypass some of the inconsistencies. An analysis of variance between academic level and modality on data of Art History students only, shows that there is no effect related to the modality, neither as simple effect (F(2,185)=2.05; ns), nor as interaction (F(4,185)=.37; ns). In Educational Sciences it is not possible to use the academic level variable because of the sparse number of students in first year Master's level. An analysis of variance using the modality variable only reveals that there is a significant difference in Educational Sciences (F(3,260)=4.55; p<.01; cf. Table 14).

Table 14. Student grade means in Educational Sciences related to learning modality

 

Table 15. Student grade means in Art History related to academic level

 

Table 14 allows a relatively similar interpretation to the one previously realized on the whole population, with one notable difference. The mean grade of on-campus students is this time higher to that of e-learning students. An analysis of variance on the whole of the sample with the factor of having followed all courses at a distance (correspondence and e-learning), reveals a significant difference (F(3,451)=11.26; p<.001). We note that this difference is mainly related to the combined modes modality as the three other means (on-campus, correspondence and e-learning) are relatively similar to each other. Post hoc tests confirm that there is a significant difference only between correspondence and combined modes students (Turkey=1.46; p<.05). Other two by two comparisons all yield non significant relationships.

Initial analysis of variance on Art History, between academic level and on-campus students variables reveals an academic level effect (F(2,188)=13.12; p<.001) and an on-campus modality effect (F(1,188)=4.04; p<.05). As Table 15 indicates, on-campus students' mean grades are better no matter their academic level. One cannot use the modality variable in this analysis because on-campus students in Art History did not combine their studies with other modes. In Educational Sciences though, data on combined modes is available. An analysis of variance with the distance modalities variable (F(1,137)=.47; ns) and on-campus (F(1,137)=1.11; ns) do not reveal any significant difference^[6]. This result indicates that in Educational Sciences, when students pass all their courses, the fact of them having taken courses at a distance (correspondence or e-learning) does not have a significant impact on their mean grades.

Discussion

The various analyses reveal that academic success interpreted at a general level is influenced by variables that may introduce distortions in a way that requires that they be subtly considered before properly inferring from data. The two most potent factors that may impact academic success for students in our sample are the academic year level and the academic discipline.

The first academic level, which is taken into account here in Art History only, has the lowest rate of students receiving credit for the academic year and the lowest mean for student grades. The fact that Educational Sciences students generally succeed better than those in Art History is explained by the fact that the former start their studies at the third undergraduate academic year, a level at which academic success is better in Art History too.

In accordance with our hypotheses, taking into account both above-mentioned variables, academic success of students who took their courses entirely at a distance is not as good as that of on-campus students. Nevertheless, analyses conducted on Educational Sciences data reveal that the difference can be reduced. The fact of combining on-campus and distance learning modes within or among courses, positively affects academic success in terms of students receiving credit for the academic year in Educational Sciences. It should be pointed out that students' mean grades in Educational Sciences who were distance learners were clearly not as affected as students in Art History. The reduced difference for the former may be explained by the effect of combined modes.

Another hypothesis that is validated by the tests we ran on Educational Sciences students, is the positive effect using a LCMS has on the rate of students receiving credit for the academic year. A higher rate of students that used e-learning passed their academic year in comparison to correspondence students in the same programs. In spite of this, similarly to the previous result, the positive effect does not show up in students' grade means, no doubt due to the same reasons as previously mentioned of absence of difference between grade means of on-campus and distance learning students.

A last hypothesis was formulated in respect to differences in the way e-learning tools are used in theoretical sciences as opposed to applied sciences. This hypothesis cannot be validated by this study as no information pertaining to the direct use of the LCMS services was collected. However, a clear difference between the two disciplines as expressing the above-mentioned categories of sciences, is revealed in terms of student academic success with the two indicators used in this study. In Art History the fact of using the LCMS for distance learning has no effect in reducing the gap with on-campus students. In Educational Sciences however, the study reveals that using the LCMS significantly raised the success rate in comparison to correspondence students.

All the analyses converge to indicate that e-learning's potential may have a positive impact on academic success for distance learners. Contrary to the illusion that predominated at the beginning of the years 2000, and that was spread by the administration of large education institutions (Palloff and Pratt 2000), the simple fact of making technologies available does not produce a virtuous effect as such. Art History students' results clearly indicate that the introduction of e-learning had no positive effect on student academic success. The opposite is true in Educational Sciences, as we noted earlier.

Be that as it may, one must not forget that different media have different impact on the contents depending on the domain. Art History relies more on images than Education Sciences, especially when it comes to representing paintings, architecture or sculpture for instance. Mediating artwork entails skills that are highly dependant on the used technologies. For Bates (1993) a distinction between media and technology is necessary. Media is a particular means for communication that is associated to a representation of knowledge. Each media therefore involves a form of representation and the organization of knowledge that is specific to it and that is tied to its format and representational mode. For example, the fade-out technique for transition between scenes in filmmaking is specific to the audio-visual media in use. A learner's interpretation of the meanings conveyed through techniques is therefore dependant on the media (Salomon & Cohen, 1977). This makes it clear that the mediation of Art History is far more sensitive to the media being used than Educational Sciences are. The interpretation of artwork is highly dependant on the media and technologies being used. As there was no specific adaptation of the LCMS to accommodate for the Art History course, it is also possible that academic success differences between the two domains in accountable to this.

In regard to the complexity of data, which can mask the existence of hidden variables, one must be weary of the ecological validity and hence the extent of these conclusions. Furthermore, end-of-year grades are no doubt not the best guarantors for assessing educational effectiveness as Carré (2005) conceives it, for example. Nevertheless, the impact that e-learning has remains coherent with the conclusions of numerous other studies, such as those presented in the introduction (Fenouillet and Déro 2006). This makes it worthy of suggesting some directions for thought in respect to bettering learning systems that wish to incorporate distance learning. Firstly, it appears worthy for educational settings relying on distance learning to incorporate at least one face-to-face course (in modular programs). This should prove beneficial for student academic success. Secondly, the effect of e-learning depends on usage. At least, it cannot be attributed only to the media as it does not manifest itself in Art History. At this level our hypothesis is similar to Clark's (1985), attributing its effect to the course design that Educational Sciences instructors used. Several explanations can be pondered. In a qualitative study on LMS use in distance learning, Jézégou (2006) notes that for instructors, innovation is not perceived as the introduction of technologies in their practices. It is held to be the skills needed to spark and favor, at a distance, the cognitive and social engagement of students in collaborative activities. Said differently, it is no doubt insufficient to technically train instructors to use educational technologies. What is required is devising the most appropriate instructional design for the particular educational goal. Finally, it comes across that some software is better adapted to certain disciplines, as underlined by White and Liccardi (2006) through the use of the notion "technological affordance". It emerges as necessary to analyze practices that facilitate knowledge construction and that develop a learning dynamic (Carré 2005) by using advantageously a set of technological tools that are adapted to these goals.

^[1]ENFASE, developed in 2003 by Moïse Déro, Fabien Fenouillet and Yves Szymczak.

^[2] The sum of individuals is slightly under 2847 because of missing data for some students, as indicated earlier.

^[3] Further on we will refer to year-level simply as level.

^[4] Combined modes students, who are only present in Educational Sciences, were grouped with on-campus students for his analysis.

^[5] Combined modes students, who are only present in Educational Sciences, were grouped with on-campus students for this analysis. Also, grades of Master level students were not used as they are only present in Educational Sciences.

^[6] Only these two modalities are accounted for.

Acknowledgements

We would like to thank Stéphane Bertolino, Jean-Philippe Quaglio and Delphine Pollet from OFIVE, plus Dominique Leconte and Nathalie Lecoq from the centre for distance learning at Lille 3 University, without whom this study would not have been possible. We would also like to thank Gérard Jean-Montcler for his fine insight and comments on the initial version of this paper.

References

[1] Albero, B. (2004). Technologies et formation : travaux, interrogations, pistes de réflexion dans un champ de recherche éclaté. Savoirs - Revue internationale de recherches en éducation et formation des adultes, 5, 11-72.

[2] Bates, T. (1993). Theory and practice in the use of technology in distance education. in D. Keegan (Ed.), Theoretical principles of distance education. London : Routledge, 213–233

[3] Beatty-Guenter, P. (2001). Distance Education: Does Access Override Success? Paper presented to the Canadian Institutional Research and Planning Association Pacific Northwest Association for Institutional Research and Planning 2001, Conference: Victoria, BC October 30, 2001.

[4] Biglan, A. (1973). The characteristics of subject matter in different academic areas. Journal of Applied Psychology, 57(3), 195−203.

[5] Carré, P (2005). L'apprenance. Paris, Dunod.

[6] Carr, S. (2000). As distance education comes of age, the challenge is keeping the students. Chronicle of Higher Education, 46(23), A39-A41.

[7] Cavanaugh, C.S. (2001). The effectiveness of interactive distance education technologies in K-12 learning: A meta-analysis. International Journal of Educational Telecommunications, 7(1), 73-88.          

[8] Cavanaugh, C., Gillan, K. J., Kromrey, J., Hess, M., & Blomeyer, R. (2004). The Effects of Distance Education on K-12 Student Outcomes: A Meta-Analysis. Learning Point Associates.

[9] Clark, R. E. (1983). Reconsidering Research on Learning from Media. Review of Education Research, 53(4), 445-459.

[10] Clark, R. E. (1985). Evidence for Confounding in Computer Based Instruction Studies: Analyzing the Meta-Analyses. Educational Technology Research and Development, 33(4), 235-262.

[11] Clark, R. E. (1994). Media Will Never Influence Learning. Educational Technology Research and Development. 42(2), 21-29.

[12] Diaz, D.P. (2000). Comparison of student characteristics, and evaluation of student success, in an online health education course. Unpublished doctoral dissertation, Nova Southeastern University, Fort Lauderdale, Florida.      

[13] Easterday, N. (1997). Distance education and 2-Year colleges. Community College Journal of Research and Practice, 21(1), 23-37.

[14] Fenouillet F., & Déro M. (2006). Le e-learning est-il efficace, Savoirs - Revue internationale de recherches en éducation et formation des adultes, 12 ,87-100.

[15] Gagne, M., & Shepherd, M. (2002). A Comparison Between a Distance and a Traditional Graduate Accounting Class. T.H.E. Journal , 28, 9.

[16] Giles, I. M. (1999). An Examination of Persistence and Dropout in the Online Computer-Conferenced Classroom. Doctoral Dissertation. Virginia Polytechnic Institute and State University, Northern Virginia Center.

[17] Haeuw F., Duveau-Patureau v., Bocquet F., Schaff J. L., & Roy-Picardi D. (2001). COMPETICE outil de pilotage par les compétences des projets TICE dans l'enseignement supérieur. Ministère de l'éducation nationale, DT, bureau B3.

[18] Jézégou, A. (2006). Au-delà du couple technologies éducatives - autonomie des Étudiants. International Journal of Technologies in Higher Education, 3(3),30-37

[19] Jones, C., Zenios, M., et Griffiths, J. (2004) Academic use of digital resources: Disciplinary differences and the issue of progression. In S., Banks , P., Goodyear, V., Hodgson, C., Jones, V., Lally, D., McConnell et C., Steeples (Eds.), Networked Learning 2004: Proceedings of the Fourth International Conference on Networked Learning 2004, Lancaster: Lancaster University and University of Sheffield, 222-229. http://www.networkedlearningconference.org.uk/. Accessed 11 April 2008.

[20] Joy, E. H., & Garcia, F. E. (2000). Measuring learning effectiveness: A new look at no-significant-difference findings. Journal of Asynchronous Learning Networks, 4, 1.

[21] Keegan, D. (1986). The foundations of distance education. London : Routledge.

[22] Liao, Y. K., & Bright, G. W. (1991). Effects of computer programming on cognitive outcomes: A meta-analysis. Journal of Research on Computing in Education, 24, 367-380.

[23] Lipsey, N. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181-1209.

[24] National Science Foundation, Division of Science Resources Statistics. (2002). The Application and Implications of Information Technologies in Postsecondary Distance Education: An Initial Bibliography , NSF 03-305, Project Director, Eileen L. Collins (Arlington, VA 2002). http://www.nsf.gov/sbe/srs/nsf03305/start.htm. Accessed 11 September 2006.

[25] Neumann, R. (2001). Disciplinary Differences and University Teaching. Studies in Higher Education, 26(2), 135-46.

[26] Omoregie, M. (1997). Distance learning: An effective educational delivery system. Information Analysis 1070. (ERIC Document Reproduction Service No. ED 418 683).

[27] Palloff, R., & Pratt, K. (2000). Making the transition: Helping teachers to teach online. Paper presented at EDUCAUSE: Thinking it through. Nashville, Tennessee. (ERIC Document Reproduction Service No. ED 452 806).

[28] Pascarella, E.T. & Terenzini, P. (1980). Predicting freshman persistence and voluntary dropout decisions from a theoretical model. Journal of Higher Education, 51(1), 60-75.

[29] Phipps, R. & Merisotis J. (1999). What's the Difference? A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education. Washington, DC: Institute for Higher Education Policy, http://www2.nea.org/he/abouthe/images/diseddif.pdf. Accessed 04 November 2008

[30] Salomon G., & Cohen, A. A. (1977). Television formats, mastery of mental skills, and the acquisition of knowledge. Journal of Educational Psychology, 69, 612–619.

[31] Spencer D. H., Hiltz S. R. (2001). Studies of ALN: An Empirical Assessment. Proceedings of the 34th Hawaii International Conference on System Sciences 2001.  http://www.hicss.hawaii.edu/HICSS_34/PDFs/CLALN05.pdf. Accessed  15 April 2007.

[32] Smith, G. G., & Ferguson, D. (2005). Student attrition in mathematics e-learning. Australasian Journal of Educational Technology, 21(3), 323−334.

[33] Smith, G., Heindel, A., & Torres-ayala, A. (2008). E-learning commodity or community: Disciplinary differences between online courses. Internet and Higher Education. 11,  152-159.

[34] Roblyer, M. D. (1999). Is choice important is distance learning? A study of student motives for taking Internet-based courses at the high school and community college levels. Journal of Research on Computing in Education, 32(1), 157-172.

[35] Valentine, D. (2002). Distance Learning: Promises, Problems, and Possibilities. Online Journal of Distance Learning Administration, 5, 3.        

[36] Vergidis, D., & Panagiotakopoulos, C. (2002). Student Dropout at the Hellenic Open University: Evaluation of the Graduate Program "Studies in Education". International Review of Research in Open and Distance Learning, 3, 2.     

[37] Waxman, H. C., Lin, M. F., & Michko, G. M. (2003). A Meta-Analysis of the Effectiveness of Teaching and Learning With Technology on Student Outcomes. Learning Point Associates.

[38] White, S., & Liccardi, I. (2006). Harnessing insight into disciplinary differences to refine E-learning design. Proceedings from 36th ASEE/IEEE Frontiers in Education Conference. San Diego, CA: ASEE/IEEE, 2006.

[39] Wirt, J., Choy, S., Rooney, P., Provasnik, S., Sen, A., & Tobin, R. (2004). The Condition of Education 2004 (NCES 2004-077). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office, 2004.