Multimodal Training of Maintenance andAssembly Skills Based on Augmented Reality

Abstract

The training of technicians in the acquisition of new maintenance and assembly tasks is an importantfactor in industry. As the complexity of these tasks can be enormous, the training of technicians toacquire the necessary skills to perform them efficiently is a challenging point. However, traditionaltraining programs are usually highly theoretical and it is difficult for the trainees to transfer the acquiredtheoretical knowledge about the task to the real task conditions, or rather, to the physical performanceof the task. In addition, traditional training programs are often expensive in terms of effort and cost.Previous research has shown that Augmented Reality is a powerful technology to support training inthe particular context of industrial service procedures, since instructions on how to perform the servicetasks can be directly linked to the machine parts to be processed. Various approaches exist, in whichthe trainee is guided step-by-step through the maintenance task, but these systems act more as guidingsystems than as training systems and focus only on the trainees sensorimotor capabilities. Due to theincreasing complexity of maintenance tasks, it is not sufficient to train the technicians execution ofthese tasks, but rather to train the underlying skills sensorimotor and cognitive that are necessaryfor an efficient acquisition and performance of new maintenance operations.All these facts lead to the need for efficient training systems for the training of maintenance andassembly skills which accelerate the technicians learning and acquisition of new maintenance procedures.Furthermore, these systems should improve the adjustment of the training process to newtraining scenarios and enable the reuse of existing training material that has proven its worth. In thisthesis a novel concept and platform for multimodal Augmented Reality-based training of maintenanceand assembly skills is presented. This concept includes the identification of necessary sub-skills, thetraining of the involved skills, and the design of a training program for the training of maintenance andassembly skills. Since procedural skills are considered as the most important skills for maintenanceand assembly operations, they are discussed in detail, as well as appropriate methods for improvingthem. We further show that the application of Augmented Reality technologies and the provisionof multimodal feedback and vibrotactile feedback in particular have a great potential to enhanceskill training in general. As a result, training strategies and specific accelerators for the training ofmaintenance and assembly skills in general and procedural skills in particular are elaborated. Here,accelerators are concrete methods used to implement the pursued training strategies.Furthermore, a novel concept for displaying location-dependent information in Augmented Realityenvironments is introduced, which can compensate tracking imprecisions. In this concept, the pointercontentmetaphor of annotating documents is transferred to Augmented Reality environments. As aresult, Adaptive Visual Aids are defined which consist of a tracking-dependent pointer object and atracking-independent content object, both providing an adaptable level and type of information. Thus, the guidance level of Augmented Reality overlays in AR-based training applications can be easilycontrolled. Adaptive Visual Aids can be used to substitute traditional Augmented Reality overlays (i.e.overlays in form of 3D animations), which highly suffer from tracking inaccuracies.The design of the multimodal AR-based training platform proposed in this thesis is not specific forthe training of maintenance and assembly skills, but is a general design approach for multimodal trainingplatforms. We further present an implementation of this platform based on the X3D ISO standardwhich provides features that are useful for the development of Augmented Reality environments. Thisstandard-based implementation increases the sustainability and portability of the platform.The implemented multimodal Augmented Reality-based platform for training of maintenance andassembly skills has been evaluated in industry and compared to traditional training methods. The resultsshow that the developed training platform and the pursued training strategies are very well suited forthe training of maintenance and assembly skills and enhance traditional training.With the presented framework we have overcome the problems sketched above. We are cheap interms of effort and costs for the training of maintenance and assembly skills and we improve its efficiencycompared with traditional training.