Framework for Digitally Mature Schools
Nina Begičević Ređep, Igor Balaban, Bojan Žugec, Marina Klačmer Čalopa, Blaženka Divjak, University of Zagreb, Croatia
Abstract
Rapid diffusion of Information and Communication Technologies (ICT) in all professional and personal areas require digital maturity from schools. In order to assess this aspect of school, a concept of digitally mature school has been developed and translated into different frameworks. In this paper, we describe development methodology of Framework for Digitally Mature Schools (FDMS) in Croatia. The FDMS, together with the accompanying instrument and software, represents a unique and comprehensive tool set for the assessment of digital maturity of a school. The FDMS recognized five areas divided into 38 elements that are described on five digital maturity levels in the form of a rubric.
Introduction
A concept of digitally mature schools is increasingly becoming significant within the modern educational system due to the growing importance of Information and Communication Technologies (ICT) in education. The European Commission has recognized the significance of this concept and, through its policies, encourages the development of digitally mature schools. We described digitally mature schools as schools with a high level of integration of ICT and systematized approach to ICT use in school management and in their educational processes. The use of ICT in schools is no longer a matter of individual enthusiasm, but a systemic approach planned and implemented at the level of school in accordance with local and state policies (e-Schools). The Framework for Digitally Mature Schools is therefore needed to enable the identification of areas and elements that contribute to the digital maturity as well as for planning of possible progress in the integration and use of digital technologies.
There are several frameworks designed regarding the digital maturity of educational institutions (Table 1). However, based on the performed qualitative analysis of these frameworks we conclude that neither of them provides a comprehensive concept which could be used as the basis for the establishment of a digitally mature schooling system. There is no concept that would encompass framework, the instrument for evaluation nor a software supporting implementation.
Based on the analysis of the existing frameworks and in line with the goals of e-Schools project in Croatia (“e‑Schools: Establishing a System for Developing Digitally Mature Schools (pilot project)” funded by the ESF and ERDF), we developed a comprehensive Framework for Digitally Mature Schools (FDMS), instrument for self-evaluation and external evaluation of schools, as well as the software supporting FDMS implementation. The FDMS consists of five evaluation areas and five levels of digital maturity and it is in line with the generic European Framework for Digitally Competent Educational Organizations (DigCompOrg). In accordance with the FDMS, the instrument for evaluation of the digital maturity of schools was prepared. The self-evaluation and external evaluation of 151 schools in Croatia was performed according to the mentioned instrument and online software. The purpose of evaluation was to determine the initial level of the digital maturity of each school included in pilot project. This was necessary in order to enable the monitoring of their progress and the planning of the most suitable means of support for schools.
Finally, in this paper, we present the methodology used in developing the Framework for Digitally Mature Schools (FDMS), as well as the FDMS itself. The methodology will not be presented in detail due to the page limit.
Objectives of the research
The overall objectives of the research within e-Schools project related to the FDMS are:
- to review the existing findings on maturity of schools, to propose main areas and elements, to describe digital maturity of schools and to build a comprehensive framework for assessment of digital maturity of schools;
- to develop a framework for evaluation of digital maturity of schools for Croatian educational context;
- to develop the Instrument (in form of a rubric and accompanying questionnaire) for self-evaluation and external evaluation of schools in order to be able to determine the digital maturity level for each school;
- to assess the level of digital maturity of schools in Croatia with self-evaluation and external evaluation using the developed instrument;
- to perform in-depth analysis of maturity levels of all schools included in the research, as well as to monitor their progress and plan the means of support within e-Schools project.
The specific objectives of this paper are:
- to briefly present methodology used in developing the Framework for Digitally Mature Schools (FDMS);
- to present developed FDMS based on theoretical findings and survey results.
Analysis of Digital Maturity Frameworks
In the scope of our research the qualitative analysis of 15 digital maturity frameworks was performed. The following Frameworks were analysed:
- Assessing the e-Maturity of your School (Ae-MoYS);
- DigCompOrg (DigCompOrg);
- eLearning Roadmap (eLearning Roadmap);
- eLemer (eLemer);
- The ePortfolios & Open Badges Maturity Matrix (ePOBMM);
- Future Classroom Maturity Model (FCMM);
- HEInnovative (HEInnovative);
- Jisc Strategic ICT Toolkit (JISC);
- Ledning, Infrastruktur, Kompetens, Användning (LIKA);
- Microsoft Innovation Framework & self-reflection tool (MICROSOFT IF & SRT);
- NACCE SRF (NACCE SRF);
- OPEKA (OPEKA);
- Up-scaling Creative Classrooms in Europe (SCALE CCR);
- SCHOOL MENTOR (SCHOOL MENTOR);
- VENSTRESS (VENSTRESS).
Within the analysis, a special attention was paid to the following elements: implemented development approach, application area, sensibility to beginning and/or advanced levels, the existence of accompanying framework, instruments for evaluating the maturity level and for the supporting software and best practice examples. An overview of the frameworks analysed in this research is shown in Table 1. The analysis revealed two frameworks/toolkits (DigCompOrg and eLearning Roadmap) that, due to their characteristics, best describe the comprehensive field of digital maturity of schools. However, further modifications and adjustments are needed for both frameworks on grounds of two major reasons: (a) to adjust the framework to the local (Croatian) context (also required and suggested by the DigCompOrg framework), (b) to update outdated elements (due to fact that the second identified framework/tool – eLearning Roadmap is outdated).
Table 1: Overview of frameworks analysis
Research methodology
The Framework for Digital Maturity of Schools (FDMS) was being developed in the period from October 2015 to June 2016 within the framework of e-Schools project by FOI (Faculty of Organization and Informatics, University of Zagreb) and CARNet (Croatian Academic and Research Network) experts. The methodological approach we used for the development of the FDMS was for the most part qualitative. It was based on the comprehensive review of academic and grey literature, a pool of the existing frameworks, meta-analysis of selected frameworks and a number of stakeholders’ consultations.
First phase
In the first phase, we have done a comprehensive qualitative analysis of 15 frameworks for the digital maturity with focus on digital technologies or some forms of digital maturity in different sectors (Section 3). Based on the results of the analysis two frameworks have been selected to form the basis for the creation of the FDMS: (a) DigCompOrg, the framework developed by the European Commission for the digitally competent educational institutions, and (b) the eLearning Roadmap tool which is very successfully used for the purpose of certification of digitally mature schools in Ireland. However, these frameworks did not cover the entire concept needed to support building of the FDMS. Roadmap is not a framework but a tool which mostly covers e-Learning. Therefore, it enables schools only to test the current level of their e-Learning maturity. Namely, the digital maturity is a broader concept than e-Learning maturity. Further, it is adjusted solely to the Irish educational system which influences its applicability in Croatia or any other country. DigCompOrg is a framework for digitally competent educational institutions and includes all the main areas of digitally competent educational institutions. Additionally, it represents a very complex and comprehensive framework that can be the basis for the assessment of all educational systems. In the development of our Framework, DigCompOrg served as a generic framework. However, its elements within the main areas were reduced and modified in order to correspond to the context of elementary and high-schools in Croatia. The described analysis was based on expert knowledge and experience. Table 2 shows the mapping of the basic dimensions of two above-mentioned frameworks on the newly created FDMS. The result of the first phase was the first version of the Framework, developed by using qualitative analysis of 15 Frameworks with focus on two indicated European models. It was followed by several demanding cycles of confirmation and revision using expert knowledge.
Table 2: Mapping of the FDMS to existing frameworks
Second phase
In the second phase of the framework development, we applied sorting cards (Q-sorting) method and two focus groups analysis as tools for defining new framework areas and their elements as well as descriptors related to the levels. Ten experts that participated in the card sorting method covered the areas of digital technologies, their application in the educational system, strategic planning and similar. The suggested pool of elements in the Q-sorting method was created on the results of the DigCompOrg analysis and on the conclusions of two focus groups performed with more than 60 principals and teachers. The new elements included in the Framework are: Assessment, Learning Analytics (LA), Content repository and licensing, Learning spaces and E-inclusion. At that stage, it was decided to base the framework on five maturity levels and to present it in a form of a rubric. Namely, the rubric enables mapping of the achievement against explicit assessment criteria. However, it is important to describe the criteria as clearly as possible.
Third phase
In the third development phase, the experts who developed the framework now determined the descriptors for all levels in rubrics form. A research was conducted, using a questionnaire, on about 70 examinees who had to prioritize the areas and elements and list new ones if they considered necessary. There were also semi-structured interviews with the representatives of school founders, ministry, school principals and digital technology experts. The participants were asked to assess the implementation of digital technology in school in order to confirm and improve the proposed areas, elements and descriptors in the FDMS. It is important to mention that, in this phase, the specificities of the Croatian system were built into the developed FDMS and Instrument. With this, the procedure of defining areas, framework elements and descriptors for all the elements on all levels was completed.
Fourth phase
Based on the FDMS, the fourth development phase resulted with the rubric (maturity matrix) for each domain with 5 maturity levels, 5 areas and 38 elements. We used mathematical (propositional) logic with logical operations and quantifiers to clearly connect statements and accurately describe maturity levels. However, since the pilot group of respondents found challenging to work with the rubrics directly, it was decided to convert the rubrics into questionnaire items. In order to map the questionnaire items into the rubrics, the use of mathematical logic proved to be valuable. In order to determine the overall maturity level of a school, Taxicab metric was used.
Fifth phase
In the fifth development phase, there were several consecutive iterations of improving the Framework and descriptor elements, the rubrics, as well as the questionnaire items (in the Instrument) with help of experts from CARNet, principals of several Croatian schools and the representatives of school founders. This resulted with the final version of the FDMS and with the Instrument that was further implemented in form of an online software. The Instrument was further validated on a sample of 151 schools in Croatia where evaluation of digital maturity was conducted.
Framework for Digitally Mature Schools
The Framework for Digitally Mature Schools (FDMS) defines the areas and levels of the digital maturity of schools. The methodology used in developing the FDMS was presented in Section 4. The schools can use the FDMS as a guide when planning and integrating the ICT in learning and teaching, as well as in their management processes. The policy creators and the decision-makers in the educational system can exploit the FDMS for the development of policies and initiatives aiming at successful integration of the ICT into the educational system. The FDMS consists of five areas and five levels of digital maturity of schools. Table 3 presents the areas and elements of the digital maturity of schools within the FDMS (e-Schools, Begicevic Redjep, 2016). Each area consists of a larger number of elements which have been described for each maturity level. In Table 4 Rubric for the element “Vision, strategic guidelines and objectives of ICT integration” is shown to illustrate the approach.
Table 3: Areas and elements of the FDMS
Table 4: Rubric for the element “Vision, strategic guidelines and objectives of ICT integration”
Below, there are brief descriptions of each maturity level. The descriptors indicate the characteristics of a typical school on a particular level of digital maturity. A specific school may differ in some aspects from a typical representative for a particular level. In the process of self-evaluation and external evaluation, each school receives the feedback based on their characteristics and regarding the assessed maturity level.
Level 1: Basic
The school is not aware of the possibility of using ICT in learning and teaching nor in management processes. Therefore, the school does not take the ICT into consideration in planning its growth and development. The ICT is not used in learning and teaching. The educational staff (teachers) do not develop their digital competences. The online communication with school is generally not possible. The ICT infrastructure has not been provided yet and the computers are used only in few classrooms in the school.
Level 2: Initial
There is awareness of the possibility to use ICT in learning and teaching and in management processes, but it has not yet been implemented. A small number of teachers use ICT in learning and teaching. There is awareness of the need to enhance the digital competences of teachers and students. However, the system for the professional development of digital competences still does not exist. The school is still inactive in the online environment and access to their own ICT resources is limited. The ICT infrastructure is generally undeveloped and computers with Internet access are available only in few classrooms in the school.
Level 3: e-Enabled
The school is aware of the possibility to use ICT in all its activities, guides the development of its strategic documents and integration of ICT into these documents. The ICT is used for working with students with special educational needs. The teachers advance their digital competences, develop digital content and start introducing innovative teaching methods. The school participates in small ICT focused projects. The access to different ICT resources is provided in most classrooms. A special attention is given to equipment maintenance and to controlling software licensing. The school is active online, in terms of content presentation and communication.
Level 4: e-Confident
The school recognizes the advantages of ICT usage in its activities very clearly and integrates the ICT implementation into strategic documents, as well as in everyday activities. The teachers use ICT for advanced teaching and assessment methods, as well as develop their own content and protect it by copyright. There is also a shared repository of content which can be used by teachers and students. The continuous professional training of teachers for the purpose of acquiring digital competences is planned and performed. Students are encouraged to develop those competences. Access to different ICT resources is provided in most classrooms, whereas the procurement and maintenance of the ICT resources is planned. The school is active with respect to ICT projects. The school is also very active online in terms of content presentation and communication. Software licensing is controlled and the security aspects of ICT use are taken into consideration.
Level 5: e-Mature
In its strategic documents and development plans, the school very clearly recognizes and requires the use of ICT in all activities. The management practice relies on the integration and obtaining the data from all school information systems. The approach to enhance digital competences of teachers and students is systematic, professional training for the teachers and additional course activities for the students are available. The teachers use ICT within advanced teaching methods, for the development of new course content and for the assessment of student accomplishments. Teachers and students regularly protect digital content by copyright. There is also a shared repository of content available for use by teachers and students. Access to ICT resources from own devices is provided in all classrooms and other rooms in the school. The school independently plans and acquires ICT resources which are available in nearly all classrooms and other rooms in the school. The entire school has a developed network infrastructure. An information security system was developed and software licensing is systematically controlled and planned. The school is characterized by varied ICT project activities, cooperation between teachers and students, as well as between other schools and stakeholders. This is done through the use of online communication tools and e-services.
Conclusion
The Framework for Digitally Mature Schools (FDMS), the accompanying Instrument for evaluation of the digitally mature schools and the supporting software developed in the scope of the e-Schools project represent a unique and comprehensive tool set created according to sound research methodology. The FDMS identifies five areas organized as a rubric with 38 elements, each described on five levels of maturity. Due to their generic characteristics, the FDMS and the Instrument can be applied in other educational systems and countries with minor adjustments. The Instrument can be used as a tool to evaluate the school’s digital maturity level but also for the identification of the areas for improvement that could enable the growth on the scale of digital maturity and improve the overall reputation and school results. The FDMS, the Instrument and the accompanying software have been already successfully applied in the process of self-evaluation and external evaluation of 151 schools in Croatia. The significant feedback for improvement of the FDMS and of the Instrument was collected in this validation process. The evaluation of further 1400 elementary and high-schools in Croatia is planned for 2017.
References
- Ae-MoYS (n.d.). Assessing the e-Maturity of your School. Retrieved January 16, 2017, from http://e-mature.ea.gr
- Begicevic Redjep, N. Et al. (2016). The Framework of Digital Maturity. Papar presented at the CUC2016 Conference, Rovinj. Croatian Academic and Research Network – CARNet. ISBN 978-953-6802-36-4
- DigCompOrg (n.d.). Digitally Competent Educational Organisations. Retrieved January 16, 2017, from https://ec.europa.eu/jrc/en/digcomporg/framework
- Europorfolio (2013). ePOBMM. The ePortfolios & Open Badges Maturity Matrix. Retrieved January 16, 2017, from http://www.eportfolio.eu/matrix
- e-Schools: Establishing a System for the Development of Digitally Mature Schools (pilot project). Retrieved January 16, 2017 from http://www.carnet.hr/e-skole
- FCMM. Future Classroom Maturity Model. Retrieved January 16, 2017, from http://fcl.eun.org/hr/toolset2
- HEInnovative. Retrieved January 16, 2017, from https://heinnovate.eu/
- Hunya, M. (2013). eLEMER. Self-evaluation of ICT usage at Hungarian schools. Retrieved January 16, 2017, from http://ofi.hu/publikacio/self-evaluation-ict-usage-hungarian-schools
- JISC. Jisc Strategic ICT Toolkit. Retrieved January 16, 2017, from https://www.jisc.ac.uk/guides/managing-course-information
- LIKA. Ledning, Infrastruktur, Kompetens, Användning. Retrieved January 16, 2017 from http://www.iktpedagogerna.se/lika-it-tempen-pa-skolan/
- NACCE SRF. Retrieved January 16, 2017 from https://www.naacesrf.com/
- OPEKA. Retrieved January 16, 2017 from http://opeka.fi/Opeka-SystemDesign-1.0.pdf
- PDST Technology in Education (n.d.). E-learning Roadmap. Retrieved January 16, 2017 from http://www.ncte.ie/elearningplan/roadmap/
- SCALE CCR. Up-scaling Creative Classrooms in Europe. Retrieved January 16, 2017 from http://is.jrc.ec.europa.eu/pages/EAP/SCALECCR.html
- MICROSOFT IF & SRT. Microsoft Innovation Framework & self reflection tool. Retrieved January 16, 2017 from http://www.is-toolkit.com/self_reflection.html
- SCHOOL MENTOR. The Norwegian Centre for ICT in Education. Retrieved January 16, 2017 from http://www.skolementor.no/index.php/en/omradeguide-en
- VENSTRESS. Retrieved January 16, 2017 from https://www.scholenopdekaart.nl/