Jendrik Bulk, Benjamin Tannert (2024). Development of a Training Simulator and Testing Environment for Electric Wheelchair Controls. In: Human Factors in Design, Engineering, and Computing. AHFE Open Access, vol. 159, pp. 1445–1452. https://doi.org/10.54941/ahfe1005712
A training simulator is being designed as part of the interdisciplinary research project “Development of alternative control systems for wheelchairs”. This simulator serves as a platform for testing and evaluating innovative control concepts that are created in an iterative process involving users with electric wheelchairs, medical and technical specialists. The development focuses on integrating the physical properties of a real, electric wheelchair into a simulation. This wheelchair is typically controlled with the hand, chin or tongue joystick and has characteristics such as inertia and acceleration behaviour. The goal is to create a driving experience that is as realistic as possible. These adjustments are continually developed in close cooperation with the groups mentioned. This project aims to clarify how the transfer of the real wheelchair into the virtual simulation affects the learning behaviour, motivation, and satisfaction of patients who complete training with electric wheelchairs.Interviews with occupational therapists at the Center for Paraplegics in Hamburg have revealed an increased need for such a training simulator. Currently, training with electric wheelchairs is limited to real-world environments, which poses certain risks. The home environment is simulated in the clinic with markings on the floor. A safer and more efficient training method would be to use a variable, virtual environment in which patients can be familiarized with the new method of locomotion without risk. For this purpose, realistic training scenarios from everyday clinical practice are integrated into the virtual environment of the training software. A virtual reality headset allows patients to adapt to using the electric wheelchair without risk and at their own pace. In cooperation with a leading manufacturer of electric wheelchairs, the plan is to use the physical hardware of the wheelchair as an interactive input device for the simulator. It will also be possible to configure the specific comfort settings (including backrest, headrest, largest, armrests) within the training simulation. In particular, the adjustability of the backrest is an important element of driving training for mastering slopes and slides.The next phase of research is concerned with the development of an eye-controlled digital joystick for the Magic Leap 2 augmented reality glasses, which will eventually also be implemented in real wheelchairs. The joystick is initially used as a control element for a virtual wheelchair in a simulated test environment. A key development goal is to visualize the wheelchair's user-specific user interface (UI) on the AR display. It is also planned to enable control of the UI via eye control and voice commands to ensure intuitive and barrier-free interaction. Thanks to flexible setting options, the UI can be personalized to the requirements of the respective user.After a comprehensive evaluation of the control by the test subjects and medical staff regarding user-friendliness and safety, the innovative control is integrated into a physical wheelchair. In a further iterative process, the control is continuously optimized in close collaboration with the end users in a protected setting.
As part of the research project “Partitive development of an alternative control method for electric wheelchairs”, a training simulator is being designed. This simulator serves as a platform for testing and evaluating innovative control concepts that are created in an iterative process involving users with electric wheelchairs, medical and technical specialists. The development focuses on integrating the physical properties of a real, electric wheelchair into a simulation. This wheelchair is typically controlled with the hand, chin or tongue joystick and has characteristics such as inertia and acceleration behaviour. The goal is to create a driving experience that is as realistic as possible. These adjustments are continually developed in close cooperation with the groups mentioned. This project aims to clarify how the transfer of the real wheelchair into the virtual simulation affects the learning behaviour, motivation and satisfaction of patients who complete training with electric wheelchairs and the possibility to train in a safe environment.
Interviews with occupational therapists at the BG Klinik Hamburg Centre for Paraplegics have revealed an increased need for such a training simulator. Currently, training with electric wheelchairs is limited to real- world environments, which poses certain risks. The home environment is simulated in the clinic with markings on the floor. A safer and more
efficient training method would be to use a variable, virtual environment in which patients can be familiarized with the new method of locomotion without risk. For this purpose, realistic training scenarios from everyday clinical practice are integrated into the virtual environment of the training software. A virtual reality headset allows patients to adapt to using the electric wheelchair without risk and at their own pace. In cooperation with a leading manufacturer of electric wheelchairs, the plan is to use the physical hardware of the wheelchair as an interactive input device for the simulator. It will also be possible to configure the specific comfort settings (including backrest, headrest, leg rests, armrests) within the training simulation. In particular, the adjustability of the backrest is an important element of driving training for mastering slopes.
NEED FOR A VIRTUAL TRAINING SIMULATOR In an expert interview with occupational therapists from the BG Klinik Hamburg Centre for Paraplegics, it became clear that more comprehensive training is necessary to teach new wheelchair users how to use their aids. After their medical treatment, those affected undergo intensive training designed to enable them to move safely in public spaces and in their homes with an electric wheelchair. Before discharge from hospital, patients must acquire certain skills and pass a type of test like a driving test. In a sports hall, real-life situations are recreated by using traffic cones, floor markings and other materials to create a course on which wheelchair driving is practiced and evaluated. Using an electric wheelchair without appropriate training can be problematic and dangerous, both for the users and those around them (Faria et al., 2014; Rodriguez, 2015). In addition to the little-changing structure of simulations, such as a street reenactment, the living environments are recreated and individually adapted for each user using floor markings. Because of the higher expanse in changing the training environment, it makes sense to think about a simulator in which beginners use their real wheelchair equipment but practice in an easy changeable virtual world. The advantages of such a simulator are:
Providing a safe training environment in which neither users nor third parties can be injured, or objects can be damaged.
A reduction in cost and time required to set up physical training environments.
Creating realistic conditions instead of using makeshift objects or markers.
The possibility of making mistakes that would have serious consequences in the real world.
Simultaneous education and training of multiple people.
This study is a segment of the overarching project “Partitive development of an alternative control method for electric wheelchairs”. The designed training simulator not only serves as a training program, but also to test and confirm innovative control techniques. It is expected that certain advantages noted above, particularly in terms of time and cost efficiency, will also apply to this application.
Development of a Training Simulator and Testing Environment 1447
Another feature that a training simulator could offer is the ability to log training and analyse various metrics. This includes, for example, the number of collisions, the distance travelled, the speed, the reaction time and the direction of view. This data could help track users’ progress, identify their weaknesses and customize training. They could also serve as feedback for users to increase their motivation and confidence.
It is important to emphasize that the simulator is not just for practicing using the wheelchair. It also offers the opportunity to practice dealing with everyday barriers through various scenarios. According to a study, the different paths one can take depend on these barriers (Tannert et al., 2019). Therefore, it is necessary to think about it in advance in order to achieve the desired goal.
IMMERSION OF THE TRAINING SIMULATOR It is crucial that the training experience offers the user a level of reality that seems as real as possible. Various studies indicate that the training effect increases with increasing level of immersion (Schenk et al., 2022; Niermann et al., 2023; Wolfartsberger et al., 2022). To achieve intense immersion, the project relies on high-poly assets and realistic textures to create realistic environments. A lot of feedback and ambient noise also contribute to immersion (Arlati et al., 2020). In particular, everyday sounds such as footsteps, engine noises, the opening and closing of doors as well as forest sounds and specifically wheelchair noises support this purpose in the software. These include engine sounds including acceleration and deceleration phases, as well as collision sounds with various materials. The user’s head movements can be transmitted to the virtual camera using a Tobii eye tracking sensor, which allows limited viewing within virtual reality. The use of virtual reality (VR) glasses is particularly suitable for a complete look around the virtual world and a significant increase in immersion (Buehler et al., 2019; Diemer et al., 2019). The possible uses of virtual reality (VR) in various training fields have proven to be advantageous. By creating a fully immersive and interactive experience, participants can fully enter the simulated world. This deep immersion not only promotes learner engagement but also increases their understanding and ability to retain information. According to one study, VR-assisted learning was considered less tiring compared to traditional on-the-job training, although conventional teaching methods were proven to lead to significantly better learning outcomes (Wolfartsberger et al., 2022). In conclusion, VR offers the opportunity to transform training in multiple areas through safe, cost-effective and immersive learning experiences, with scientific studies emphasizing both the benefits and the need for accurate design of VR learning environments to ensure optimal learning outcomes (Zak et al., 2022).
A technically important implementation of the training software is the integration of a joystick from a real wheelchair. The prerequisite for this is that the current wheelchair can establish a Bluetooth connection with a computer, so the integrated hardware can be used to control the virtual wheelchair.
1448 Bulk and Tannert
FUNCTIONAL SCOPE OF THE SOFTWARE The simulation software is being developed iterative in collaboration with experts in this field, like occupational therapists from the paraplegic centre in Hamburg, therapists from the personal environment and technical staff from the wheelchair manufacturer Ottobock. Working together has proven successful. The inclusion of medical and technical expertise promotes the necessary change of perspective for development work.
The simulated wheelchair is based on the Ottobock Juvo B5 mid-wheel model provided by the manufacturer. Like its real counterpart, it has five adjustable maximum speeds. The reversing speed can also be reduced in percentage depending on the selected maximum speed and regulated manually while driving.
Adjusting the seat angle is crucial for wheelchair users to be able to manoeuvre safely on different inclines; If the angle is incorrect, a user could slip out of the seat. The software prevents movement if the seat is not adjusted correctly. In addition, the behaviour during braking and acceleration throttling can be simulated and individually adjusted depending on the driver load to ensure a realistic digital driving experience.
Features that increase immersion, as discussed in an earlier chapter, are added to deepen the user’s interaction with the virtual environment. For example, activating the horn causes pedestrians to stop and make room for the wheelchair user. If the wheelchair is parked on the side of the road with the hazard lights on, passing cars will stop until it has passed the road.
Seat operation, including rotation, as well as the under-development eye controls, can be activated and deactivated via voice commands. Gears can also be changed using voice, and there is the option of bringing a rear-view camera into view using a voice command. Figure 1 shows the operator interface, which is overlaid on the user view in the right upper corner and remains invisible in the virtual reality view. The interface displays configurable voice commands as well as control-specific commands for the operator. In addition, a bird’s eye view camera in the lower left corner provides the operator with additional visual monitoring options.
Figure 1: Endless scene with no obstacles to get familiar with the controls.
Development of a Training Simulator and Testing Environment 1449
The application has three ready-made training scenarios. The first is a world without any obstacles and can be navigated endlessly (Fig. 1). Here the user should have the opportunity to familiarize themselves with the controls without having to take driving behaviour and obstacles into account.
The second scenario resembles a section of a small residential area (Fig. 2). It includes options to add obstacles such as dumpsters and illegally parked vehicles.
Figure 2: Residential area with pedestrians and cars.
It is also possible to show pedestrians and moving cars walking around. The houses in this scene are furnished and can also be driven into.
In the third scenario, users face a series of arcade-style obstacles designed specifically to challenge the wheelchair’s manoeuvrability and potentially knock it off the platform (Fig. 3). This playful element aims to improve users’ driving skills.
A ranking list shows the speed with which the participants completed the course. This provides wheelchair users with feedback on how they have performed compared to their previous attempts and how they compare to other users.
Figure 3: Game environment to train and challenge the user.
CUSTOM LEVEL DESIGN A special function of the application is the ability to design your own virtual environments. Users can choose from various options to customize their levels according to their ideas and needs. For creative users there is an editor that allows free design. The application comes with some simple primitive
1450 Bulk and Tannert
forms from which the training scenarios from real training can be translated into the digital world. Users can also import their own 3D models without the need for a software update. It is also possible to program your own functions, like moving obstacles for example, for the new environments, but not necessary. The scope of the editor offers many options, especially for experienced users. However, it can also easily be used by less experienced users. Typical training scenarios are provided as examples that users of all experience levels can use as a guide and to start with.
To recreate the patients’ homes or another familiar environment, the so- called home designer was integrated (Fig. 5). Here it is possible to recreate the floor plan of the patient’s apartment and then furnish it in a 3D environment with virtual furniture. The software can for example configure the width of the doors and set the opening direction of the doors what is important for wheelchair users, to train how to handle it. To make it easier to get started, there are also scenes prepared for this level designer that can serve as a template or inspiration.
Figure 4: Scene editor for custom level design.Figure 5: Level designer to recreate patients homes. Upper left: furnishing the home, lower left: drawing of the floor plan, settings for door sizes.
Development of a Training Simulator and Testing Environment 1451
OUTLOOK The next phase of research is concerned with the development of a digital joystick for the Magic Leap 2 augmented reality glasses, which will eventually also be implemented in real wheelchairs. The joystick is initially used as a control element for a virtual wheelchair in a simulated test environment. A key development goal is to visualize the wheelchair’s user-specific user interface (UI) on the AR display. It is also planned to enable control of the UI via eye control and voice commands to ensure intuitive and barrier-free interaction. Thanks to flexible setting options, the UI can be personalized to the requirements of the respective user.
After a comprehensive evaluation of the control by wheelchair users and medical staff regarding user-friendliness and safety, the new control is integrated into a physical wheelchair. In a further iterative process, the control is continuously optimized in close participatory development with the end users in a protected setting.
It is also planned to implement virtual wheelchairs with other drive types such as front and rear drive, so that the significantly different driving behaviour can also be trained with these wheelchairs.
CONCLUSION
The work has presented a virtual training simulator for electric wheelchair users that provides a safe, cost-effective and intensive learning experience. The simulator is the result of interdisciplinary collaboration between wheelchair users, occupational therapists, wheelchair manufacturers and software developers. The simulator allows users to train and improve their driving skills in various scenarios tailored to their individual needs and goals. The simulator uses the wheelchair’s existing hardware as an input device and offers various immersion factors that make the training experience more realistic and motivating. The simulator also offers the ability to design your own levels to recreate users’ homes or other familiar environments. In addition, the simulator serves as a platform for testing and evaluating innovative control concepts that are being developed in another research project. The work shows how virtual reality can transform and optimize the training of electric wheelchair users by creating a flexible, personalized and immersive learning environment.
REFERENCES
Arlati, S. et al. (2019) ‘Virtual reality-based wheelchair simulators: A scoping review,’ Assistive Technology, 32(6), pp. 294–305. https://doi.org/10.1080/10400435. 2018.1553079 Buehler, K. and Kohne, A. (2019) ‘Lernen mit Virtual Reality: Chancen und Möglichkeiten der digitalen Aus- und Fortbildung,’ in Springer eBooks, pp. 209–224. https://doi.org/10.1007/978-3-662-59527-5_11 Diemer, J. and Zwanzger, P. (2019) ‘Die Entwicklung virtueller Realität als
Expositionsverfahren,’ Nervenarzt/Der Nervenarzt, 90(7), pp. 715–723. https: //doi.org/10.1007/s00115-019-0678-6
1452 Bulk and Tannert
Faria, B. M. et al. (2014) ‘A survey on intelligent wheelchair prototypes and simulators,’ in Advances in intelligent systems and computing, pp. 545–557. https://doi.org/10.1007/978-3-319-05951-8_52 Niermann, P. F.-J. and Palmas, F. (2023) ‘Die Zukunft ist hier: Schlüsselkompetenzen
mit Virtual Reality trainieren!,’ in Springer eBooks, pp. 113–143. https://doi.org/ 10.1007/978-3-658-37901-8_8 Rodriguez, N. (2015) ‘Development of a wheelchair simulator for children with multiple disabilities’ in 3rd IEEE VR International Workshop on Virtual and Augmented Assistive Technology (VAAT), Arles, France, 2015, pp. 19–21, https: //10.1109/VAAT.2015.7155405 Schenk, J. et al. (2021) ‘Knowledge management in virtual worlds: Knowledge generation in virtual reality environments,’ HMD. Praxis Der Wirtschaftsinformatik, 59(1), pp. 159–176. https://doi.org/10.1365/s40702-021- 00814-z Tannert, B., Kirkham, R. and Schöning, J. (2019) ‘Analyzing accessibility barriers
using Cost-Benefit analysis to design reliable navigation services for wheelchair users,’ in Lecture notes in computer science, pp. 202–223. https://doi.org/10. 1007/978-3-030-29381-9_13 Wolfartsberger, J. et al. (2021) ‘Virtual Reality als Trainingsmethode: Eine Laborstudie aus dem Industriebereich,’ HMD. Praxis Der Wirtschaftsinformatik, 59(1), pp. 295–308. https://doi.org/10.1365/s40702-021-00819-8 Zak, K. and Oppl, S. (2021) ‘Der Einsatz von Virtual-Reality Lernszenarien für Softskills-Trainings,’ HMD. Praxis Der Wirtschaftsinformatik, 59(1), pp. 142–158. https://doi.org/10.1365/s40702-021-00784-2
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Design, Engineering, and Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-35-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional