With this letter from szenaris, we would like to inform you of some interesting facts and news from the world of e-learning. In this issue with we will inform you of the competence-oriented learning with Digital Twin Simulations on the Vivatop research project and on the potential forms of learning using extended reality technologies. We will also give you a report on the guest seminar at the University of Applied Sciences (HAW), were szenaris CEO Bock-Müller held a speech on “Virtual Training Systems”.
We hope you enjoy reading our newsletter.
Developed for you…
Digital Twin Solutions: expertise-oriented learning with digital twins
Representation of the Digital Twin Simulation for the training of operating personnel of bridging systems
In the course of digital development, the technological concept of the so-called Digital Twin is causing a stirr across industries. A digital twin is defined as the digital representation of a process, procedure, object or system from the real world in virtual space. This results in numerous possible applications, for example for production processes. In the virtual world, developers can change functions or test new settings, which in reality would involve considerable effort.
But the use of digital twins also holds great potential for the design of expertise-oriented learning processes. In the Digital Twin Simulation, developed by szenaris for the training of operating personnel for bridging systems, the systems Amphibie M3, Folding Floating Bridge (FSB), and Folding Fixed Bridge (FFB) were transferred into the virtual world as 3D models, with the aid of CAD data. Their behaviour is simulated with all applicable physics. The vehicle and system functions are emulated realistically. The essential improvement to the simulations used so far for the training is that the behavior of the digital twin, in relation to defects occurring on the vehicle or system due to failure of sensors or drives, for example, corresponds to the original. This way, the operator learns to recognize technical problems which may occur and, if possible, to solve them. The operator can provide maintenance with precise fault descriptions, and maintenance can practice repair work on the digital twin.
The simulation is a multi-user VR application, i.e. different actors interact with each other in a virtual 3D world. The virtual reality glasses used allow for “immersion” in this world. Various control devices are available, such as controllers and data gloves, as well as the original operating devices integrated into the simulation. The degree of reality and the interaction of the components thus ensure that the simulation is perceived as extremely real, which enables the seamless transition to the application of what has been learned virtually into reality, and considerably shortens the actual training period. In addition, the parameters of the virtual environment can be changed and malfunctions simulated. The structure is scalable so that additional complete workstations of an amphibious vehicle can be integrated. The application can take place at different locations and, if the data connection is available, the workstations or the trainer workstation can be located at different locations.
Digital Twin Simulations thus enable expertise-oriented training of operating and maintenance personnel, and significantly support the learning processes even independently of the original system.
Contact: Klaus Bock-Müller
Researched for you …
Vivatop: The Future of medecine – VR, AR and 3D-printing for surgeries
New information technologies can help to provide surgeons in the operating theatre with important information, and thus noticeably increase the chances of success of the operation. A research network led by the Technology Centre for Computer Science and Information Technology (TZI) at the University of Bremen is currently developing exemplary applications to improve the planning and execution of an operation through the use of Virtual Reality (VR), Augmented Reality (AR) and 3D printing. The results will also be used for training purposes and patient information. The project VIVATOP (Versatile Immersive Virtual and Augmented Tangible OP) is funded by the Federal Ministry of Education and Research (BMBF) with a total of 2.2 million euros over a period of three years. From a medical point of view, there are three potential areas of application in everyday clinical life: the preoperative phase, the intraoperative phase, and training and education situations.
The planning of an operation with 3D models offers considerable advantages over the 2D images that have been widely used so far. Doctors gain a much more realistic overview of the area in which they want to perform an operation. This enables them to precisely prepare the individual steps. Parallel to the three-dimensional computer models, VIVATOP is also testing the provision of lifelike physical models of the affected organ. This involves importing the image data into a 3D printer so that, for example, a patient’s liver can be reproduced realistically.
These printed models enable the physician to experience the individual organ haptically during planning, i.e. using their sense of touch, but they can also be used together with the virtual models. In this way, someone can hold the physical model in their hands and gesticulate in an explanatory manner, while all participants on the VR glasses can see the gestures in connection with the highly detailed computer model. This method of operation is more in keeping with natural habits than with controllers, such as those used in game consoles to control movements. Meanwhile, the 3D prints not only facilitate planning, but also enable the medical team to explain the procedure to the patients more clearly.
During the operation, the organ’s image data helps doctors to orient themselves and make decisions. For this purpose, the VIVATOP team is developing an augmented reality application, which displays the relevant information in the field of view of glasses. Voice control allows surgeons to operate the device without having to use their hands. At the same time, it is possible to display the operation live in VR so that external experts can be interactively consulted outside the operating theatre (“remote, collaborative, immersive surgeon consultation”). With the help of depth cameras, sensors and a bundle of other state-of-the-art technologies, the hand movements of the team of doctors as well as the interventions on the operated organ are recorded and displayed in VR. The planning data may also be displayed in parallel in the VR world. This way, the system makes it possible for specialists to work together worldwide during an operation.
The performance of medical interventions requires extensive theoretical and practical skills as well as many years of experience. Training and practical testing are traditionally carried out according to the “See One, Do One, Teach One” method, i.e. learners first observe experienced doctors and later carry out operations themselves under their guidance. However, this type of training requires enormous personnel and time resources and also entails a high risk for the safety of patients.
With the integration of AR, VR and 3D printing, the training and further education of young doctors can be made more efficient. Concrete cases can be discussed using the 3D image data and precisely reproduced in virtual reality. The printed 3D models allow young doctors to gain their first valuable experiences. For example, they can learn to feel a tumour at different stages of its development (“haptic feedback”) and then examine the situation in a VR environment. In addition, individual surgical steps can be trained in virtual scenarios or real operations can be followed by telepresence.
The VIVATOP project is coordinated at the TZI of the University of Bremen by the Digital Media Working Group (Prof. Dr. Rainer Malaka) and supported by the Department of Virtual Reality and Computer Graphics (Prof. Dr. Gabriel Zachmann). These teams are primarily responsible for VR interaction and VR algorithms. The Fraunhofer Institute for Image-Based Medicine (MEVIS) produces the image data, while apoQlar GmbH as a specialist in augmented reality and CIRP GmbH for 3D printing are involved. The szenaris GmbH is responsible for training and education and the University Hospital Oldenburg participates as application partner. The project is scheduled for completion in September 2021.
Contact: Dr. Uwe Katzky
On the road for you …
szenaris holds guest seminar at the University of Applied Sciences (HAW) in Hamburg
Impressions from the seminar
At HAW, students in the “Media Systems” program are qualified for a career in computer and network technology, especially in the media and games industries. The media industry places high demands on technology: storage of large amounts of data, processing in real time, transmission of multimedia applications in heterogeneous networks, and much more. In addition to IT-related training, students also acquire interface skills in design, audio and video production technology, and the use of these media.
In the second year, students specialise in computer science and electronics, networks, IT security, databases, software engineering, image processing, and virtual systems. As part of the lecture “Virtual Systems” held by Professor Dr. Neuhöfer, szenaris CEO Klaus Bock-Müller held a guest seminar focusing on “Virtual Training Systems” for the second time. The seminar, which was attended by 30 students, included four topic blocks:
At the beginning, an overview of areas of application and industries with practical examples was given. Furthermore, from a technical point of view, the topics Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR), taken together as so-called Extended Reality (XR), were considered more closely. Subsequently, the students developed a so-called strengths-weaknesses-opportunities-risk analysis (SWOT) as a group task for the “introduction of virtual training in companies”.
At the beginning of the second block Klaus Bock-Müller presented the topic “Conceptual planning of virtual training systems” using the example of driver training for the new Wuppertal suspension railways and deepened the task, the analysis and the implementation concept. A second task for the group work was the compilation of checklists supporting the realization of virtual training systems. Aspects to be considered included organization, technology, project and resource management, as well as the goal of successfully completing the project and obtaining an acceptance protocol from the customer.
Before and after the event, as well as during the break, the students had the opportunity to use a practical example with a wireless VR training system consisting of VR glasses, VR gloves and an operating device to lay a mobile bridge in a mixed reality simulation. Thus, all participants can look back on an all-round successful event, which gave the students an in-depth insight into the various topics of virtual training systems, and which had provided many interesting and instructive conversations.
Contact: Klaus Bock-Müller
Read up for you …
Extended Reality (XR) – Learning in the Future
Versatile applications for learning with Extended Reality
Learning with digital media has undergone a rapid development in recent years. This pace is expected to continue in the coming years. But where does this journey lead us? What will learning look like in the future?
For us at szenaris GmbH, it is clear that the importance of knowledge as a stockpile is losing importance, while the development of self-organised (action) competences is moving to the fore. Training systems that either enable the application of what has been learned in a virtual, realistic environment or extend reality by virtual aspects, are particularly suitable for this purpose. The collective term for these technological possibilities is “Extended Reality” (XR).
Extended Reality generally refers to all technologies that combine virtual environments with reality. These include, for example, virtual, mixed and augmented reality applications. How to learn efficiently with these forms is briefly outlined in the following.
Virtual Reality” (VR) refers to realistic computer simulations. What is important here is the degree of immersion, i.e. the feeling of being completely immersed in the virtual world. “Admission tickets” are 3D glasses (VR glasses), which allow a stereoscopic perspective, and controllers or hand scanners. These allow learners not only to passively participate in the digital world, but also to act in it themselves. The use of VR in learning scenarios offers a number of advantages. Thus, rare situations such as emergencies can be trained without risking danger and damage to people and equipment. However, this is only justifiable up to a certain ethical limit, because the mistakes, which have no consequences in the VR world, can have serious consequences in reality.
In Augmented Reality (AR), unlike VR, learners do not immerse themselves completely in virtual reality, but rather add additional information to the real world through displays or AR glasses in the form of texts, images, animations or the like. Some systems also allow the extension of reality by means of voice output. The fields of application for AR in education are manifold. For example, when carrying out service or repair work, the necessary action steps can be displayed on the real device as an overlay at the respective viewing angle. Learning content can thus be called up on demand.
A mixed form is the so-called “Mixed Reality” (MR). The conceptual differentiation between VR and AR is not always possible. In general, a mixed reality system can be defined as all those systems, in which the position of the learner in the room is recorded and transmitted into the virtual world (so-called position tracking). Here parts of the real world are integrated into the virtual reality. An example of a learning process that can be designed with MR competence-oriented is the performance of work actions in virtual production facilities.
As specialists for learning with digital media, we at szenaris GmbH are following developments in the field of extended reality with great excitement in order to be able to offer our customers innovative, efficient and individual solutions that enable competence-oriented learning processes in motivating scenarios.
Contact: Malte Rathjen