Methodology for Design of Web-based Laparoscopy e-Training System
Daniela Borissova*[dborissova@iit.bas.bg],
Ivan Mustakerov [mustakerov@iit.bas.bg],
Institute of Information
and Communication Technologies of Bulgarian Academy of
Sciences [http://www.iict.bas.bg],
Department of
Information Processes and Systems,
Acad. G. Bonchev St., Block 2,
1113 Sofia, Bulgaria
Abstracts
Bulgarian
Съвременните мултимедийни инструменти, съчетани с възможностите на Интернет, позволяват на Уеб-базираното електронно обучение да преодолява много от недостатъците на традиционното обучение. Статията е фокусирана върху използването на виртуална мултимедийна среда за разработване на електронна система за повишаване нивото на знания и практически умения на студентите по медицина и на практикуващите лекари в областта на миниинвазивната хирургия. Описан е концептуален модел на уеб-базирана медицинска информационна система, включваща методология за електронно обучение, както и модел на връзките в учебното съдържание. Системата допринася за поддържане и усъвършенстване нивото на знания и практически умения на специалистите по миниинвазивна хирургия, запознавайки ги с най-новите научни постижения и съвременните лапароскопски технологии. Описан е изследователски прототип на уеб-базираната медицинска информационна система за електронно обучение по миниинвазивна хирургия.
English
The Web-based e-learning can benefits from the modern multimedia tools combined with network capabilities to overcame traditional education. The objective of this paper is focused on e-training system development to improve performance of theoretical knowledge and providing ample opportunity for practical attainment of manual skills in virtual environment. A conceptual design Web-based medical information system including an e-training methodology and a model of content relations is described. The system helps the practitioners to keep up to date with the latest scientific breakthroughs and state-of-the-art laparoscopy techniques. The developed prototype of Web-based medical information system for laparoscopy e-training is illustrated.
Keywords
Web-based information system; e-learning, e-training, medical information system; design methodology.
Introduction
ICT-based learning as interactive and collaborative provides more enriching and motivating learning environment both within and outside the classroom. These new learning systems also allow learning to be personalized to user needs in terms of both content and method. The advances in ICT offer a lot of advantages for significant developments in the sector of health care, affecting the medical diagnosis, patient and healthcare management, and especially medical education. There is nowadays an international trend to involve computers and the Internet in medical curricula, as well as, in continuing life-long medical learning and training. One of the main challenges of contemporary teachers and trainers refers to the design and implementation of new training processes which have to meet the challenges and necessities of the digital age. Currently the enormous expansion in medical and biomedical knowledge poses a fundamental challenge in medical education. In order to account the over-specialized knowledge available by different experts, information technology could be employed to develop new pools for educational modules and provide the mechanisms and tools to combine individual learning styles. Using of e-learning enhance traditional learning, support existing teaching methods and provide a valuable reference point which can be accessed anytime, anywhere.
One of the important changes in medical practice over the past decades has been inherent in the surgical interventions. The new surgical and interventional approach called laparoscopy or minimally invasive surgery has become the standard of care for many routine diagnostic and therapeutic procedures (appendicitis, gallstones, hernia, hyperhidrosis, gastro-esophageal reflux, etc.). The laparoscopic revolution brings also new demands for training and gaining experience and with the current technological advancements a new frontier of training is needed. The traditional education of laparoscopy has a number of shortcomings, which do not allow demonstrating procedures of surgical operations, surgical technique, instruments, examination methods and the performance of exemplary demonstrations of paraclinical examinations. Teaching materials are insufficiently focused on practical management of the given topics and there is an insufficiency of photographic documentation and documentary videos. Furthermore, the fast developments in medical device technology arises the needs of new and updated teaching materials to meet the key competencies defined in health sector including the needs of in-service training Çetin (2010).
The Web-based technologies utilize the learning activities through enhance of knowledge and performance by highly dynamic multimedia presentations delivered via hypermedia e-learning environments. A multidimensional concept model of ICT enabled classroom to highlight potential similarities and differences between where teachers perceive themselves relative to their learners (Beyers 2009). The comprehensive usability of the e-learning platform concerning a design is evaluated by innovative approach in education (Granića and Ćukušić 2011).
To keep up to date with the latest scientific breakthroughs, current medical debates and state-of-the-art medical technology it is needed online or off-line e-learning recourses. The e-learning resources offer a vast amount of possibilities in different medical fields tailor-made for a variety of users – students as well as trained physicians, employee workers or managers. The publications in the field of medical e-learning could be categorized in terms of reported themes usage by: type of study; type of technology; type of learner; subject area; learning; use of videoconferencing; guidelines and general trends (Lau et al. 2004; Wutoh et al. 2004; Blavier et al. 2007).The laparoscopic training program should provide possibilities for acquiring of important new set of skills to get full advantage of the minimally invasive surgery technique. Effective teaching and learning involves modern teaching instruments such as laparoscopic trainers, virtual simulators. The laparoscopic training and simulator-based curricula have proven effective in providing skills that translate to the operating room and are an excellent cost-effective training option (Keyser et al. 2000; Scott et al. 2000; Aggarwal et al. 2006; Stefanidis et al. 2006; Kanakala et al. 2010; Sroka et al. 2010; Thijssen et al. 2010), nevertheless the virtual reality simulators cannot yet predict levels of real life surgical skills (Seymour et al. 2002).
Recent advances in information and communication technologies contribute to the health care sector improvement affecting strongly medical diagnosis, patient and healthcare management and especially medical training. Advanced ICT is a prerequisite to e-training software and make it a necessity to enforce the skills needed for both basic and advanced procedural training. At this time, the rapidly evolving technology of Web-based e-training represents a disruptive technology that requires realignment of traditional methods of surgical training into new paradigms of knowledge acquisition (Hasson 2004).
The current paper proposes a methodology for design of Web-based laparoscopy surgery e-training system conforming to SCORM recommendations and latest research results in the area. This methodology identifying the stages and needed activities for laparoscopy surgery e-training development and describe the distributed roles between types of specialists – medical experts and ICT specialists. The proposed conceptual model of Web-based laparoscopy systems introduce potential abilities to provide e-training platform on which medical users can practice and learn skills.
Specifics of e-training
E-learning development from the basic use of ICT for learning to new forms of education and training, emphasize on creativity and collaboration and require new skill for the knowledge society. This requires a significant change of accent, away from a focus on technology, connectivity and the Internet, towards a greater consideration of the context of learning, and of the need for collaboration, communication and innovation. The effectiveness of e-training technology on learners depends not only on what outcomes are targeted but also on how the technology is integrated into instruction. Mixing the different media tools contribute to more effective training and identifying the most adequate learning styles combinations as described in several studies (Wang et al. 2011; Duan et al. 2007; Karakaya et al. 2007; Zhang et al. 2006; Jones et al. 2003). The learning motivation and interest is positively and strongly correlated with perceived comprehension and satisfaction, furthermore – the learning interest is more influential in improving the process than learning motivation (Hung et al. 2011).
The surgery training requires a balance of teaching theoretical knowledge and providing ample opportunity for practical attainment of manual skills. Surgical residents do not always have adequate opportunities during their medical residency to observe, discuss, access and manage the full pathological range related to their specialties. Sometimes, the direct procedures observation may be limited or unfeasible due to restrictions related to limited space. Modern imaging and communication technology allows the performance of medical procedures to be visualized in real time or off-time presentations of procedures been performed.
The rapid growth of new surgical procedures and reduced physician teaching time results in an increasing need for effective alternatives from which surgeons can gain skills. For healthcare professionals, quality of e-learning activities is critical for ensuring patient safety and increasing productivity. The e-learning content includes text and multimedia materials that are provided to learners. There exists a collection of set of technical standards for e-learning software products – SCORM. The SCORM's recommendations show how the learning content has to be designed including evaluating existing content, analyzing potential audiences, designing sharable content objects, designing aggregations and designing content structure. All SCORM content should be Web-deliverable and all SCORM communication occurs within the context of a Web browser session. The SCORM API requires communicating with an API object provided by the LMS responsible to cater educational, administrative, and deployment requirements. The LMS accepts any SCORM content and makes it available to users. The learning content development stages involves: evaluating existing content, analyzing of the potential audiences, design of sharable content objects and their aggregations and design content structure by using multimedia tools for basic elements description and e-learning building principles.
Design stages of Web-based surgery e-training methodology
One of the main concerns of contemporary teachers and trainers refers to the design and implementation of new training processes to meet the challenges and necessities of the digital age. Taking into account the minimally invasive surgery specifics and SCORM recommendations, a Web-based methodology for laparoscopy e-training to increase the level of knowledge and practical skills of medical students and practitioners can be structured as shown on Figure 1.
Figure 1. Methodology stages
The stages of the proposed methodology are described bellow.
Analysis of the laparoscopy training specifics and the shortcomings of the traditional teaching methodology
The first stage is related to the specifics and analysis of the laparoscopic surgery e-training considering of the shortcomings of the traditional teaching methodology. The minimally invasive surgery training requires a balance of teaching theoretical knowledge and providing ample opportunity for practical attainment of manual skills. Surgical residents do not always have adequate opportunities during their medical residency to observe, discuss, access and manage the full pathological range related to their specialties. For example, sometimes the direct procedures observation may be limited or unfeasible due to restrictions related to limited space. Using of modern imaging and ICT allows the performance of medical procedures to be visualized in real-time or off-real-time presentations. In this respect more laparoscopic videos, operative procedures, laparoscopic pictures, slides and proper quizzes should be included into development training methodology.
Design of the multimedia Web-based technology for laparoscopy e-training
The second stage includes design of the multimedia Web-based e-training framework technology from the software programming point of view. Taking into account all of the analysis considerations and recommendations of the above activities a framework of multimedia Web-based technology for minimally invasive surgery e-training can be developed. It should include all of the data requirements, networking restrictions, multimedia solutions, etc., for the server-side and client-side modules together with hardware specifications to implement it successfully.
Development of multimedia technology for laparoscopy e-training
When developing a multimedia e-training technology for laparoscopic surgery the recommendations for the technology assessment model are to be used. The main requirement for easiness of use can be satisfied by developing of friendly and intuitive interface, information thoroughness and multi-input access to it. Another important feature of the designed laparoscopy e-training system is the possibility to use it on different computer platforms and the flexibility to modify and extend the learning content. Those considerations should be taken into account when developing a multimedia technology for laparoscopy e-training.
Development of the laparoscopy learning content – texts, pictures, videos, audio, animation, etc.
The next stage is developing of the learning content taking into account the potential audience (students, post-graduates, and practitioners). Using of multimedia components can enliven presentation of factual information and offers many advantages when used for e-training. It enhances the text only presentations by adding interesting sounds and compelling visuals, improves over traditional audio-video presentations done with slides or overhead transparencies, gains and holds attention – combination of aural and visual presentation offers greater understanding and retention of information and is great entertaining educational tool. The videos, texts, pictures, slides, quizzes, simulation, etc., are important factors identifying impinge on the successful integration of training tools into a laparoscopy e-training program. E-testing system is based on proposed conceptual approach for development of educational Web-based e-testing system (Mustakerov & Borissova, 2011). The information system content comprises all instructional material, which can range in complexity accordingly to the potential users from discrete items to larger instructional modules. A digital learning object is defined as any grouping of digital materials structured in a meaningful way and tied to an educational objective. Learning objects represent discrete, self-contained units of instructional material assembled and reassembled around specific learning objectives, which are used to build larger educational materials such as lessons, modules, or complete courses to meet the requirements of a specified curriculum. Examples include tutorials, case-based learning, hypermedia, simulations, etc. Content authors use instructional design and pedagogical principles to produce learning objects and instructional materials. The SCORM's recommendations allow designing the learning content including different content structure considerations. During the design, content structure the multimedia tools for basic elements description and e-learning building principles are defined.
Reviewing and certifying the content for laparoscopy e-training system
Medical e-training content modules designs conform to medical students and/or practitioners needs. Postgraduate and continuing medical education differ from undergraduate education in knowledge and skills to improving physician competence and performance in practice. It is extremely important to use reviewed and certified medical learning content. That should be done by proven professionals and specialists in the relevant medical area. The e-training courses should pass regulatory compliance in healthcare documented requirements. As the learning content is one of the most valuable and critical components of an e-training development the reviewing and certifying of it is an essential activity of the current project. As a result a certified learning content for laparoscopic surgery will be available.
E-training modules programming and implementing of the laparoscopy learning content
When choosing the proper programming tools the technical implementation of SCORM should be taken into account. The programming process of software modules follows some known steps for software design:
- Conceptual design – the important task in creating a software product is extracting the requirements or requirements analysis. The users typically have an abstract idea of what they want as an end result, but not what software should do.
- Coding level – the programming code effectiveness could be improved by not allowing a bad quality programming and frequently correcting of the errors.
- Portable code – the portability is important for e-training software. It means that users can use it independently of their computer platform (operating system generally). When assuming the browser environment the operating systems differences are not the critical but the different browsers itself usually support different subsets of the programming standards.
The developed learning content is loaded in the relevant data storage places (databases). The Web-based e-learning considers allocation of the databases on the server-side.
Testing of the developed laparoscopy e-training system
The primary purpose for software testing is to detect failures in operation so that defects may be corrected. That activity includes examination of code as well as execution of that code in various environments and conditions as well as examining the different aspects of code. The result of the activity completion will be finding of software bugs and correcting them properly. The testing process is usually repeated several times until a sufficient confidence for code correctness level is achieved. That activity includes evaluation of the developed multimedia methodology for laparoscopic e-training – is the information presented in a clear manner, is it factual, is it been verified from a primary information source, is it concise and useful, etc.
Conceptual model of Web-based e-training content relations
The architecture of e-testing system includes a learner interface, which facilitates access to the e-training contents. Whilst the didactic aspects not pay much attention to technical issues, they now come into play. The various units appearing in the e-training systems outline have to be analyzed and integrated, so each unit can be supported by a combination of some data content with some functionality. The interface should support all the features that are available in existing state-of-the-art e-learning and e-knowledge tools. In addition, the e-training courseware includes different content of topics and themes realized by exercises, quizzes, tests and simulations. All of these e-training facilities are based on texts, images, presentations, audio and video data as shown on Figure 2.
Figure 2. Conceptual model of e-training content relations
The conceptual model of e-training content relations involves different type of learning contents that provide the flexibility of the system and make the learning process accessible to the wider world. All of these contribute to enhance the courseware capacity in complex and dynamic environments. The conceptual model of e-testing system content relation aims to improve the quality and effectiveness of education and training system and to open education and training to the wider audience. The proposed model for Web-based laparoscopy e-training could be implemented as a part in different Web-based learning management system. Based on the Web service, this approach could increase the efficiency and effectiveness of collaborative learning in terms of e-learning standards, reusability, interoperability, accessibility, and modularization.
Management of e-training courseware development
The implementation of the proposed methodology requires a design team consisting of members with different skills and roles. All of these responsibilities are divided in five e-training courseware phases – analysis, design, development, implementation and evaluation as it is shown on Figure 3.
Figure 3. Management of e-training system development
The individual team members' roles are distributed between two types of specialists – medical experts and ICT specialists. In general, the medical team members are responsible for all activities concerned to creating, certifying and evaluating of the medical learning content. The ICT team members are responsible for the activities from the scope of information technologies.
The project manager is a medical expert with essential practical and teaching experience in laparoscopy. It will be best if he/she has some knowledge about the ICT tools and their possibilities and applications. The project manager is involved in coordination and control of all e-training development stages. The ICT coordinator should be experienced software developer in the area of Web-applications. He/she is responsible for implementation of ICT methods and tools in software realization of the laparoscopy e-training system and also takes part in all design stages. The medical experts and content authors are highly skilled laparoscopy practitioners with teaching experience. They are main responsibility is creating of the certified laparoscopy training content on stages E and F. The software programmers are specialized in Web-applications coding. Their responsibilities include software modules and learning management system development, learning content implementation and functional testing of the system – stages G and H. The multimedia producers are Web designers whose expertises include creating of multimedia content and its implementation in the developed software modules – stage E. The system administrator is a person responsible for network application (Web-server) and managing of laparoscopy e-training system – stages G and H. One of the most essential team members responsibility concerns the stage of e-learning content creation. The team members' involvement during the e-training courseware phases (analysis, design, development, implementation and evaluation) development includes the following activities.
The analysis phase of project, the coordinator, medical experts and content authors are focused on identification of requirements toward the required outcomes and existing learning content resources. The ICT coordinator also takes part in that phase to be clear about the requirements of the content implementation.
The design phase is driven by the outputs of the analysis phase and learning objective, plan assessment instruments and devise learning activities are defined. It this phase, the software programmers and multimedia producers join to develop the structure content and requested media. As a part of the design phase a graphical user interface template for layout and navigation is also developed. The content structure includes the relationships among the elements of the content and defines the multimedia e-training technology to be implemented.
The development phase focuses on the creation and assembly of the content that is identified in the design phase. The SCORM content can be developed by any authoring tool and programming language. The content structure diagram should be defined and used to program the sequencing rules in a functional prototype with generic content. The prototype is used to test the sequencing rules before the actual content development is completed. This will help to ensure that the sequencing rules will work properly when the actual content is ready to be deployed. During the development phase a comprehensive quality assurance is done to ensure that the content is error free, accurate, functional and compliant.
The implementation phase focuses on deploying the content in the e-training environment to ensure that the learner data is been recorded as it is intended. The software programmers and system administrator are also involved in the learning content implementation. The implementation phase ends when the content is fully deployed.
The evaluation phase collects feedback to improve learning content and to maximize the training effects. Collecting data throughout all phases allows seeing where content may need to be revised. Evaluating throughout content development lifecycle will be done by all team members using peer, user and learner feedback to detect any flaws in the training materials.
GUI design
The developed and tested prototype of the Web-based medical information system for laparoscopy e-training involves the proposed methodology and the content model e-training relations. The functionality of the e-training systems supports the navigation through the learning material. The medical information system for laparoscopy e-training incorporate the horizontal and vertical navigation. The horizontal menu navigation covers the e-trading system themes, virtual simulation & exercises, quiz-base tests & on-line exams, glossary and help, as is shown in Figure 4.
Figure 4. Е-training system graphical user interface (GUI)
The vertical navigation emphasize on particular theme components following by theme overview, learning outcomes, text documentations, video training films, image gallery, laparoscopic equipment, publications, scientific forums and related links.
This system has two basic modes – guest (student) and administrator modes. Using the guest mode is possible to learn the existing materials and testing the user's knowledge by using the virtual simulation and exercises as well the quiz-base tests and on-line exams. The administrator's mode allows updating this medical information e-training system.
The development of medical information e-training system contents is based on using normal HTML to XML. XML has been accepted as the standard for Web applications developing providing much richer online experience. In order to facilitate the interaction between database and Web-page content development, the scripting language AJAX is used.
The proposed approach is based on modular architecture; therefore the additional modules could be easily integrated. The selection of employed modules depends completely on user needs. Each of these modules can be accessed separately, as they are independent of one another. The users can interact with the medical information system for e-training trough Web browser by choosing different menus navigations (Figure 4).
The experimental results demonstrated that the implementation of the proposed approach is quite helpful and facilitate the users' understanding using such Web-based e-training system application. The testing showed the practical applicability of the described medical information system for e-training development. The simple and intuitive GUI was very well accepted by the students and practitioners.
Conclusions
This paper is focused on e-training system development to improve performance of theoretical knowledge and providing ample opportunity for practical attainment of manual skills in virtual environment. The described conceptual design for a Web-based medical information system for e-training involves an e-training methodology, a conceptual model of content relations and GUI for laparoscopy e-training system design. The proposed approach could be useful for management Web-based medical information systems. The information e-training system is based on modern ICT network and multimedia capabilities for creation of educational material increasing the training effect. It assist to accumulate the over-specialized knowledge available by different experts in the field of the laparoscopy. Mixing the different multimedia tools contribute to more effective learning and to combine the most adequate learning styles combinations. As a result of the proposed methodology the drawbacks of the traditional teaching methods would be overcame by increasing the level of knowledge and practical skills of laparoscopy students and practitioners. Implemented as Web-based learning framework, the power of Internet environment give the advantages in an easy-to-use system that brings the educational experience to the foreground and puts technology in the background. The testing showed the practical applicability of the described medical information system for e-training development allowing the practitioners to keep up to date with the latest scientific breakthroughs and state-of-the-art laparoscopy techniques. The simple and intuitive GUI was very well accepted by the students and practitioners. The proposed medical information system facilitate personalized delivery of contents based on the individual user's knowledge and learning preferences.
* Corresponding author: Department of Information Processes and Systems, The Institute of Information and Communication Technologies at Bulgarian Academy of Sciences, Acad. G. Bonchev, Str., Bl. 2, 1113 Sofia, Bulgaria. Tel.: +395 29792055, Fax: +359 28707273.
References
- Aggarwal R., T. Grantcharov, K. Moorthy, J. Hance, A. Darzi. (2006): A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. The American Journal of Surgery. 191, 128-133.
- Beyers R. N. (2009): A Five Dimensional Model for Educating the Net Generation. Educational Technology & Society, 12(4), 218-227.
- Blavier A., Q. Gaudissart, G. B. Cadière, A. S. Nyssen. (2007): Comparison of learning curves and skill transfer between classical and robotic laparoscopy according to the viewing conditions: implications for training. The American Journal of Surgery. 194(1), 115-121.
- Çetin A. (2010): 3D Web Based Learning of Medical Equipments Employed in Intensive Care Units, Journal of Medical Systems, DOI 10.1007/s10916-010-9456-5
- Duan R., M. Zhang. (2007): Design of Web-based Management Information System for Academic Degree & Graduate Education. Integration and Innovation Orient to E-Society Vol. 2, IFIP Advances in Information and Communication Technology, Vol. 252, 218-226, DOI: 10.1007/978-0-387-75494-9_27.
- Granića A., and M. Ćukušić. (2011): Usability Testing and Expert Inspections Complemented by Educational Evaluation: A Case Study of an e-Learning Platform. Educational Technology & Society, 14 (2), 107-123.
- Hasson H. M. (2004): New Paradigms in Surgical Education: Web-Based Learning and Simulation, 13th Int. Congress and Endo EXPO a Simulation Course, http://www.laparoscopytoday.com/2004/01/new_paradigms_i.html
- Hung Shin-Yuan, Kuo-Liang Huang, Wen-Ju Yu. (2011): An empirical study of the effectiveness of multimedia disclosure of informed consent: A technology mediated learning perspective. Information & Management, 48(4-5), 135-144.
- Jones D., S. Gregor, T. Lynch. (2003): An Information Systems Design Theory for Web-based Education, IASTED International Symposium on Web-based Education, Rhodes, Greece. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.105.658&rep=rep1&type=pdf (accessed August 2010).
- Kanakal, V., S. Bawa, P. Gallagher, S. Woodcock, S. E. Attwood, L. F. Horgan, K. Seymour. (2010): Outcome of patients in laparoscopic training courses compared to standard patients. Surgeon 8(3), 132-135.
- Karakaya A. F., Ş. T. Pektas. (2007): A Framework for Web-based Education Systems Supporting Interdisciplinary Design Collaboration. METU JFA 2007/2, 24(2), 137-148.
- Keyser E. J., A. M. Derossis, M. Antoniuk, et al. (2000): A simplified simulator for the training and evaluation of laparoscopic skills. Surgical Endoscopy, 14, 149-53.
- Lau F., J. Bates. (2004): A Review of e-Learning Practices for Undergraduate Medical Education. Journal of Medical Systems, 28(1), 71-87.
- Mustakerov I., D. Borissova. (2011): A conceptual approach for development of educational Web-based e-testing system. Expert Systems with Applications, 38(11), 14060-14064.
- Saadé R. G. (2003): Web-Based Educational Information System for Enhanced Learning, EISEL: Student Assessment. Journal of Information Technology Education, Vol. 2, 267-277.
- Seymour N. E., A.G. Gallagher, S. A. Roman, et al. (2002): Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Annals of Surgery, 236, 458-463.
- Scott D. J., P. C. Bergen, R. V. Rege, et al. (2000): Laparoscopic training on bench models: better and more cost effective than operating room experience? Journal of The American College of Surgeons, 191, 272- 83.
- Sroka G., L. S. Feldman, M. C. Vassiliou, P. A. Kaneva, R. Fayez, G. M. Fried. (2010): Fundamentals of Laparoscopic Surgery simulator training to proficiency improves laparoscopic performance in the operating room – a randomized controlled trial. The American Journal of Surgery. 199, 115-120.
- Stefanidis D., R. Sierra, J. R. Korndorffer, Jr., J. B. Dunne, S. Markley, C. L. Touchard, D. J. Scott. (2006): Intensive continuing medical education course training on simulators results in proficiency for laparoscopic suturing. The American Journal of Surgery. 191, 23-27.
- Thijssen A. S., M. P. Schijven. (2010): Contemporary virtual reality laparoscopy simulators: quicksand or solid grounds for assessing surgical trainees? The American Journal of Surgery. 199(4), 529-541.
- Wang M., D.Vogel, W. Ran. (2011): Creating a performance-oriented e-learning environment: A design science approach. Information & Management, doi:10.1016/j.im.2011.06.003
- Wutoh R., S. A. Boren, E. A. Balas. (2004): eLearning: A Review of Internet-Based Continuing Medical Education. Journal of Continuing Education in the Health Professions, 24(1), 20-30.
- Zhang D., L. Zhou, R. O. Briggs, J. F. Nunamaker, Jr. (2006): Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness. Information & Management, 43(1), 15-27.