Pen or Keyboard – An Empirical Study on the Effects of Technology on Writing Skills
Benedetto Vertecchi, Antonella Poce, Francesco Agrusti, Maria Rosaria Re, Università Roma Tre, Italy
Abstract in Italian
Il presente studio, realizzato nell’ambito del modulo “Metodi e tecniche di scrittura in educazione” attivo presso l’Università Roma TRE, parte dall’assunto in base al quale gli studenti possono sviluppare le loro capacità di scrittura e pensiero critico grazie a specifiche attività dedicate allo scopo. Tali attività hanno riguardato, in particolare, l’elaborazione di saggi brevi, effettuati sia a mano, sia su tastiera, per verificare la differenza nei risultati ottenuti in base alla tecnologia adottata. Lo sviluppo del pensiero critico è alla base di altre ricerche del Laboratorio di Pedagogia Sperimentale – Università Roma TRE (eg. Poce, 2012), dove lo studio di autori classici in un percorso di apprendimento strutturato ha mostrato elaborazioni critiche e personali dell’apprendimento. L’obiettivo generale del modulo nel quale si è realizzata la sperimentazione è identificare un modello da poter applicare in diversi contesti e situazioni. Meta obiettivi sono stati identificati nell’opportunità per gli studenti di migliorare il corretto uso della lingua (grammatica, morfo-sintassi, accuratezza lessicale, capacità argomentative e sviluppo del pensiero critico e della creatività al temo stesso). I lavori degli studenti sono stati valutati dai ricercatori LPS, usando strumenti di valutazione appositamente sviluppati. I risultati vengono presentati nella versione estesa del contributo.
State of the art
As stated in the scope of the conference “The social and socio-economic context is more important than ever. Society itself can be understood as a learning environment, with questions of learners’ connection with the community and the empowerment of the practitioners”. This means that certain skills, the so called 21st century skills, must be prompted and developed. Writing skills are of paramount importance in this context and, as educators and researchers, we are primarily engaged in understanding the effect the use of mobile devices has on such abilities.
The relation between digital tools and writing skills regards different scientific fields, such as education, neuroscience and sociology. Everyone, teenagers in particular, is used to communicate through smartphones, tablets or PCs, writing short text messages. However, in these cases, writing skills are not developed in an efficient and productive way even if they are practiced every day. International studies and research highlight a drastic reduction of argumentation capabilities in writing for people who make too much use of texting.
Research carried out by Drew Cingle and Shyam Sundar (2012) from Penn State University (USA) has shown that an excessive use of short text messages compromises grammar accuracy and linguistic skills in high school students. Jennifer Myers (2012) from Calgary University states that reading on paper and using traditional tools for writing promote a more creative use of the language; this does not happen with digital writing. A study by Cambridge Assessment (Suto, 2012) carried out among 633 university teachers states that writing skills have been considered at the top of university students’ problems.
Even if certain advantages are present due to almost synchronous communication, at the same time a strong limitation on the potentialities of both written and oral language use occurs. Crystal (2001) talks of an Internet language made of a mixture of oral and written speech, which has an impact on learning and on certain skills in particular.
In 2012, a conference entitled Handwriting in the 21st century? An educational summit has been organized in Washington (USA). During the summit, it was highlighted that the obligatory use of keyboard in primary school writing activities generates problematic issues. In the USA, youngest generations face difficulties when they have to read handwritten texts. In relation to this, the conference indicated the need to change this situation and reintroduce handwriting as a necessary activity for the development of learning, argument and critical thinking skills.
Also neuroscience is interested in the effects of handwriting. Studies in the field highlight differences in learning among students who use digital tools to write and students who write by hand (Longcamp et al., 2008, 2011; Spitzer, 2013). Handwriting influences reading comprehension because it activates different and specific parts of the brain: handwritten character recognition happens in connection with handwriting muscle movements, providing, at the same time, visual receptors and memory capacity (Longcamp et al., 2011). In China, for instance, it has been shown that computer writing severely restricted primary school students reading skills. In one of the most recent reports by OECD, Students, Computers and Learning. Making the Connection (2015), for the first time, the Organization state that better results in reading are shown by those pupils who make less intensive use of technology in learning.
Within the above context, this contribution refers to the activity carried out at Roma Tre University, where a group of students’ writing skills have been tested, taking into account critical thinking skills levels as a referential criteria. Assignments foresaw the use of pen or keyboard on different controlled situations. Data and results were analysed accordingly and compared as described below.
Research design and methodology
The present study carried out within the module Writing Methods and Techniques in Education, hold out at University Roma TRE – Laboratory of Experimental Pedagogy (LPS), starts from the assumption that students can develop their writing and critical thinking skills thanks to specific writing activities. Such activities regard the elaboration of short essays, both by hand and on computer keyboards, with the aim to highlight the gap in results. The development of critical thinking skills is at the basis of other LPS researches (Poce, 2012) where the study of classical authors in a structured learning path has shown development in the students’ personal and critical elaboration of knowledge.
The general objective of the module where the experimentation took is to improve students’ writing skills in different disciplinary and learning contexts. Meta-objectives have been identified in the opportunity for students to improve also their correct use of the language (grammar, morpho-syntax, lexicon accuracy), argument skills and to develop critical thinking and creativity skills as well.
Over the lecturing, students have produced short essays on the topics discussed with the lecturer. Assignments were marked by LPS researchers, using an ad hoc essay assessment grid. Short essays written by students were produced in two different ways: by hand or keyboard. All data have been collected and analysed to highlight the different results in skills development according to the writing tool employed.
In the following paragraph a description of the assessment tool based on critical thinking level indicators is given.
Assessment tool
A short essay is a semi-structured test in which students have to present their ideas following a given structure. The main purpose of this kind of test is to present a set of ideas on a given topic following the order that makes most sense to a reader; therefore, the focus of an essay is its linear and logic structure.
The initial thesis provided in the guidelines, the limited time and space frame and the attention to the typical essay structure represent close stimuli, which the student must use in order to organize the development of the proposed topic. In particular, the guidelines contain a thesis, a series of questions the student has to answer, the sources and data given to develop and support his/her personal thesis and a predetermined text length. The presence of these stimuli facilitates the production of more homogeneous tests and, consequently, an easier use of an essay assessment grid.
Within this project, a specific assessment grid was outlined to evaluate students’ essays. In a short essay, one’s verbal ability is used in a specific way, that is in relation to the correct use of language and argumentation of a specific topic, and not in a general way. In this way, the teacher is able to evaluate the students’ critical acquisition of knowledge and how they use this knowledge in their essays. Moreover, the acquisition of specialized vocabulary will be clear and immediate. In relation to this, the essay assessment grid contains a macro-indicator of Basic linguistic skills, thanks to which the evaluator can assess linguistic form of the text.
As for other macro-indicators, Justification assesses students’ ability to elaborate on their thesis and to support their arguments, throughout their short essay. An essay is a specific kind of text that contains all the information readers need to know in the order in which they need to receive it; the ability to produce written argumentation on a given topic is paramount to write a good essay.
Relevance is a macro-indicator that analyses text consistency, such as the correct use of outlines and students’ capability to accurately use given stimuli. Therefore, short essays involve several different operations: introducing the argument, analysing data, raising counter arguments, concluding; this indicator assesses the correct text structure.
The Importance macro-indicator assesses the knowledge students use in their essay; asking students to write an essay on a specific topic is a good way to assess the bulk of their study.
Finally, Critical evaluation and New ideas are macro-indicators that analyse students’ critical thinking skills, by assessing personal and critical elaboration of sources, data and background knowledge with the use of new ideas and solutions associated with the initial hypothesis and student’s personal thesis.
The macro-indicators presented in the assessment grid have been selected to help evaluators during the evaluation phase: indicators, descriptors and marks are provided in detail for an easier use of the grid and to allow a reliable test evaluation.
The above essay assessment grid was used by LPS researchers to evaluate short essays written by the students engaged in the "Writing Methods and Techniques in Education" module active in the Education course at Roma TRE University.
Analyses and findings
The evaluation of the students’ short essays has shown an improvement as for all the skills connected with Critical Thinking macro-indicators and students’ writing ability itself. The table below highlights results by each student (A, B, C, etc.) participating in the module on each test and their results are measured on a scale from 5 (Clearly Insufficient for all the indicators) to 30 (Excellent in every indicator). In general, students have improved their ability to write texts in various disciplinary and learning contexts.
Table 2: Activities and results
As shown by the comparison between the first and the last test, most students have improved their writing skills: test score has increased from 1 point (e. g. student D) to 13 points (e. g. student C).
Only in the case of student B results test worsened in the last test compared to the first: however, as shown in the table above, student B did not complete all the learning path activities and wrote only 3 short essays out of a total of 6. For this reason, student B’s performance cannot be taken into account for the overall evaluation of the project.
In general, the trend of the last test score is good and 4 students out of a total of 16 (25%) obtained an excellent mark (27-30 points).
Figure 1. Comparison of First test and Last test
In particular, handwritten short essays got higher scores than computer written texts: collected data shows that short essay written on keyboard usually worsen the student’s performance over the learning path.
If we take the case of Student C, as shown in the following tables, she/he has significantly improved her/his performance during the project, going from an unsatisfactory first test (17 points) to an excellent final test (30 points); it must be noted that all short essays written on keyboard obtained a lower score than previous and following tests, probably affecting his/her learning path trend.
Even if the third test score of Student A is higher than the second one (second test: 20 points; third test: 21 points), it reduces the learning path performance improvement. Student A’s test scores increase on average by 4 points, except in tests written on PC: third test (1 point) and fifth test (-3 points).
Figure 2. Students A and C results in all the activities
A more detailed comparison between the score of the third keyboard test and the fourth handwritten test shows different performances for each student: in most cases, the fourth test has been marked with higher scores than the third one. In some students’ performance, the difference in marks is very high between the third and the fourth test (e.g. Student I: 20 points in third test vs. 29 points in fourth test; Student P: 18 points in third test vs. 26 points in fourth test).
All fourth test scores are passing (good) scores and 5 students out a total of 13 (38%) obtained an excellent score. On the other hand, 11 third test performances are lower than fourth test ones.
Figure 3. Comparison of third test (keyboard) and fourth test
Conclusive remarks and possible developments
The figure below represents a synthesis of the trend of the essay tests performed. Results for keyboard tests are clearly identifiable.
Figure 4. Comparison of all the activities for group of students
From the analyses carried out and the results shown in the figure above, some main considerations can be singled out:
- · the project learning path aims at successfully developing correct use of the language (grammar, morpho-syntax, lexicon accuracy), writing and argument skills as well as critical thinking and creativity;
- · the use of PC in writing activities reduces the performances of the majority of students.
Therefore, the experience within the project demands researchers to further develop such a model further, also extending its application to other fields of knowledge. It would be, in fact, interesting to observe if this model, which asks students to reflect through writing short essays, analysed in the above mentioned modalities, promotes critical thinking and written production, even in a context where, traditionally, it is not common to read and write about topics based on essentially humanistic and pedagogical features.
References
- Cingle, D., & Sundar, S. (2012). Texting, Techspeak, and Tweens: The Relationship between Text Messaging and English Grammar Skills. New Media & Society, May 11. Retrieved from http://nms.sagepub.com/content/early/2012/05/10/1461444812442927
- Crystal, D. (2001). Language and the Internet. Cambridge, UK: OUP.
- Longcamp, M., Boucard, C., Gilhodes, J. C., Anton, J. L., Roth, M., Nazarian, B. et al. (2008). Learning through Hand or Typewriting Influences Visual Recognition of New Graphic Shapes: Behavioural and Functional Imaging Evidence. Journal of Cognitive Neuroscience, 20, 802-815.
- Longcamp, M., Hlushchuk, Y., & Hari, R. (2011). What differs in Visual Recognition of Handwritten vs. Printed Letters? An FMRI Study. Human Brain Mapping, 32, 1250-1259.
- Myers, J. (2012, February 17). Texting Affects Ability to interpret Words. [Blog post] Utoday. Retrieved from http://www.ucalgary.ca/news/utoday/february17-2012/texting
- Poce, A. (2012). Contributi per la definizione di una tecnologia critica: un’esperienza di valutazione, premessa di Vertecchi B. Milano: FrancoAngeli.
- Ritchel, M. (2011, October 22). A Silicon Valley School That Doesn’t Compute. [Blog post] New York Times, Technology. Retrieved from http://www.nytimes.com/2011/10/23/technology/at-waldorf-school-in-silicon-valley-technology-can-wait.html?_r=0
- Spitzer, M. (2013). To Swipe or not to Swipe? The Question in Present-day Education. Trends in Neuroscience and Education, 2(3-4), 95-99. Retrieved from http://www.sciencedirect.com/science/journal/22119493/2/3-4
- Suto, I. (2012). What are the Impacts of Qualifications for 16 to 19 Year Olds on Higher Education? A Survey of 633 University Lecturers. Cambridge Assessment. Retrieved from http://www.cambridgeassessment.org.uk/Images/116010-cambridge-assessment-he-research-survey-of-lecturers-executive-summary.pdf