Iveta Eimontaite, Sarah Fletcher, Krystian Goławski, Tomasz Kołcon (2023). Breaking the barriers: multilingual user engagement to increase process engagement and technology acceptance in manufacturing. In: Human Aspects of Advanced Manufacturing. AHFE Open Access, vol. 80, pp. 1–8. https://doi.org/10.54941/ahfe1003503
With the start of Industry 5.0, there is greater emphasis on increased workforce sustainability. Manufacturing among other industries realised the economic importance not only of increased production efficiency, but the positive impact physical and psychological workforce wellbeing has on the company. The current paper presents a three-step approach of engaging multicultural end users for robotic technology introduction in the manufacturing where language dependent knowledge capture is challenging. The first step is video analysis of the process to determine which human factors might be key contributors to the existing processes. The second proposed step is process observation while the operators wear eye tracking glasses combined with several questions for the process clarification. This step allows to determine decision making points and visual attention sequence. Finally, a focus group conducted with small group of representative operators. The paper will introduce the use cases and protocol to achieve a two-fold aim: (i) feedback to the technology developers and engineers, the user critical aspects of the existing aspects, and (ii) to increase user acceptance and engagement with the developing technology/processes. The user acceptance and engagement with the final solution is expected to be improved due to the proposed three step engagement program delivered at the start of the project.
With the start of Industry 5.0, there is greater emphasis on increased workfo- rce sustainability. Manufacturing among other industries realised the econo- mic importance not only of increased production efficiency, but the positive impact physical and psychological workforce wellbeing has on the company. However, to achieve this potential, innovation and changes needs to be acce- pted by the end-user and the new processes need to correspond to user needs and requirements.
Automation and robotics have been part of manufacturing from the 70s, however, with the advancement of collaborative robotics, they are recently more often being deployed without barriers between human operators and other machines. These changes have an impact in the workforce emotional wellbeing and in significant changes on the operational processes (Gualtieri et al., 2020). Yet, to explore how these changes form the human factors point of view, a deeper understanding of existing processes is needed as well as the ability to foresee and prevent the potential difficulties onto the assembly pro- cesses. Research has established the importance of user centred research for the user increased acceptance (Pais et al., 2021) with task analysis often being the first step for in-depth understanding of the existing process. This analy- sis allows to capture human tacit and procedural knowledge of the existing process to determine necessary changes and improvements, i.e. innovation opportunities in the workplace. However, increasing collaboration not only between disciplines and industries, but also cross culturally, introduces the challenges of capturing language dependent knowledge.
Task analysis is one of the methods used to capture tacit knowledge and cognitive steps that otherwise might be hard to document. This has been shown to be efficient to achieve a deeper understanding of the process and human cognitive steps involved (Tofel-Grehl & Feldon, 2013), as well as being successfully deployed in manufacturing (Johnson et al., 2019). Hiera- rchical task analysis also has been successfully adapted specifically to adjust and plan manufacturing tasks for human robot interaction (Tan et al., 2010). However, this method heavily relies on spoken language and operator and researcher verbal interaction during the process, thus making it difficult to apply non-native speakers, and exploring multicultural environments.
Surprisingly, task analysis being deployed to investigate technology adva- ncements is relatively uninfluenced by technological development of the research tools. There is limited research showing how task analysis can be enhanced by tools used in research. One example being eye tracking technology. Eye tracking was successfully used to explore tacit knowledge in Japanese pottery making, and inform about tacit knowledge gained with increasing experience (Nakamura & Nagayoshi, 2019). Furthermore, postural position tracking and head ergonomics was a complimentary tool deepening the understanding of expert skills in refinery patrol inspection (Takamido et al., 2022). Using these techniques, visual tools allow a play- back of the process which accompanied with the operators can enhance the knowledge and understanding of the process (Postlethwaite et al., 2022). Exploratory work conducted with seamstresses on the sowing process using eye tracker capture and playback technology allowed participants not only to indicate what the steps were but provide further elaboration on why these steps were necessary (Postlethwaite et al., 2022). These examples indicate how technology can enhance task analysis, and how it can complement rese- archers’ understanding in case verbal communication might be with its own limitations.
Understanding the process and engaging in the discussion of what the cur- rent process is as well as the current needs and challenges is important, yet it does not guarantee operator inclusion in generating the solutions and greater
Breaking the Barriers: Multilingual User Engagement 3
new process ownership. Integration of the potential users into the generation of the solution results in increased acceptance and technology engagement levels (Shore et al., 2018). Such engagements, once again, are not only hea- vily language dependent, but different participants knowledge and experience background pose a challenge to communication. Workshops relying on visual tasks as well as taking participants out of their comfort zones by asking them to draw or accomplish tasks by using LEGO, can serve not only as useful sou- rce of information, but warm up participants for more open discussion about the challenges they are facing as well as potential technology solutions they would like to see (Gwilt et al., 2018). This method relies on gamification of the process which in a medical setting has been shown to increase user enga- gement (Cechetti et al., 2019) as well as self-efficacy and being in control (Borghouts et al., 2021).
The current paper presents a three-step approach of engaging multicultu- ral end users for robotic technology introduction in the manufacturing. The first step is video analysis of the process to determine which human factors might be key contributors to the existing processes. The second proposed step is process observation while the operators wear eye tracking glasses com- bined with several questions for the process clarification. This step allows to determine decision making points and visual attention sequence as well as provide the final list of the human and cognitive factors involved in the process. These factors then are investigated further with psychometric questi- onnaires. Finally, a focus group conducted with small group of representative operators. The activities in this engagement are focused on the discussion of the emerging issues from step one and two, and generation of solutions which could be taken by technology developers and integrated to the developing solutions.
The current work is a collaboration between two European projects - CONVERGING and AI-PRISM focusing overall on nine industrial use cases. The paper will introduce the use cases and protocol to achieve a two-fold aim: (i) feedback to the technology developers and engineers, the user critical aspe- cts of the existing aspects, and (ii) to increase user acceptance and engagement with the developing technology/processes. The user acceptance and engage- ment with the final solution is expected to be improved due to proposed three step engagement program delivered at the start of the project.
METHODOLOGY
The following section presents one use case analysis as an example of the pro- cess conducted within step 1 (video analysis) and step 2 (Eye-tracker guides cognitive task analysis).
Participants Two male participants from an industrial partner assembly line for small ele- ctronics took part in this study. One participant had 4 years of experience on this process, while the other participant was performing the assembly for the first time, however, had previously observed and is familiar with the assembly
steps. Participants native language was Polish, yet, one participants did not speak English while the other was proficient in English language.
Research Ethics This research was approved by the Cranfield University Research Ethics Committee, and conducted in accordance with the Cranfield Research Inte- grity Policy, the British Psychological Society’s Code of Human Research Ethics, and the General Data Protection Regulation 2018.
Task The assembly tasks consist of precise positioning of small semi-finished ele- ctronic components (diameter of 1-2mm) against a wire (Figure 1). The positioning is performed manually by using a microscope and measuring sof- tware to ensure the precision of the assembly. Once positioned in place the wire is then glued to the electronic component. The same procedure (precision positioning and precision gluing) is performed for each electronic component and each last around 5 minutes depending on the experience of the operator.
Procedure The use case lead assisted the researcher for the interpretation in the parti- cipant’s native language. Upon reading the informed consent and agreeing to take part in the data collection, the operators had a demonstration of the eye tracking glasses and further explanation about how they work. Partici- pants were asked to wear the eye tracking glasses during the assembly and to verbalize the steps they were performing. The researcher asked questions regarding the process to gain more in-depth knowledge regarding what the operators were doing and why. The observations lasted about 50 minutes for each operator (expert operator assembled six electrical components, and novice operator three during this time). After the assembly was completed, each operator was invited for a semi-structured interview.
Figure 1: The electronic component assembly.
Breaking the Barriers: Multilingual User Engagement 5
Observations Participants were asked to verbalize what they are doing during the assem- bly. The use case lead was assisting in translating the clarifying and probing questions from the researcher and answers from the operator.
Semi-Structured Interviews The researcher asked the main five questions and then depending on the answer asked follow-up questions. The structured questions were: 1. What is the most difficult during the assembly; 2. Where potential errors can occur and how operators fix them; 3. What variation of the assembly does occur (novice vs experienced operators); 4. How long does the training take to be confident in the assembly? 5. Which aspects of the tasks are the most enjoyable?
RESULTS
The heatmaps of the fixation times on three areas of interest: monitor used for calibrating the microscope, microscope, and the component provides some insight on how the process was approached and completed by two operators. The novice operator had a more spread fixation on the environment and wider dwell range compared to the experienced operator who was focused only on particular areas of the AOIs (Figure 2). Interestingly, the numerical comparison of the dwell times of these two operators indicate that although the novice operator had larger dwell times in all three AOIs (Figure 2C), the difference was the most evident in the microscope AOI.
These results combined with the follow up interviews of the operators to provide an opportunity to explain their eye tracker results provided some insight on how the assembly is approached by different experience operators. The novice operator explained that he was looking more at the microscope than he expected due to difficulty in manipulating the adjustments and positi- oning the lens. Furthermore, he indicated that looking on the computer screen (monitor) he was not always certain if he was looking at the right place and therefore the gaze was more spread out. The experienced operator on the other hand was twice as quick as the novice operator and more importantly his gaze focused on only certain aspects of the screen, while the microscope was manipulated without paying visual attention to the adjustments (tactile
Figure 2: Eye tracker results: heat map of dwell time for (A) novice operator, (B) expe- rienced operator. (C) Plotted statistics of dwell time (%) in these AOI as a function of operator experience.
information from the environment). Both operators indicated that the most mental focus was on computer screen as it allows to determine if the electro- nic component is placed correctly and if the metal wire is be glued properly. The more experienced operator explained that although the task itself is not difficult, the process requires a great level of precision and responsibility as if errors are made, there is no way to fix them – the electronic component gets scraped.
The discussion with the operators allowed to determine that during the assembly situation awareness and mental workload might be impacted depending on automation steps introduced. Furthermore, although physical discomfort is minimal according to the operators, the current process might have influence on the eye strain due to long periods of working in front of the computer screen and great focus on the precise details. Finally, the operators being in charge of the process and, in particular, the experienced operator having his own routines (i.e. pre-arranging components ready for the wire to be glued, and then gluing several at the same time) can have an impact on the perception or job responsibilities and roles - job satisfaction and self-efficacy are organizational factors to be tracked throughout technology introduction.
FUTURE WORK
The initial phase or user engagement allowed to determine which human, cognitive and organizational factors can have impact in the automation intro- duction in this use case. The following step of the process is development of the workshop to engage operators and explore what solutions could be made for the existing uncomfortable aspects of the process. As the operators are non-native English speakers with varying degrees of English proficiency,
Breaking the Barriers: Multilingual User Engagement 7
the workshops are being designed to rely on minimal verbal information and provide hands on activities with visual props to encourage the development of the solutions. The developing workshops are being based on work con- ducted for developing visual signage (Eimontaite, 2022; Gwilt et al., 2018) and using LEGO to increase stakeholder buy-ins (de Saille et al., 2022). This approach has previously been successfully used for being the intermediary between robotics engineers and shopfloor operators, resulting in greater user acceptance of the new processes (Eimontaite et al., 2022) as well as an impact on greater trust in human-robot interaction (Cameron & Collins, 2021). Thus, the expected outcomes of the engagement programme is: (i) increased level of user engagement and acceptance of the new processes; (ii) increased technology readiness levels and greater trust in automation, and (iii) more transparency in the changes of the processes and assembly resulting in greater organizational commitment.
CONCLUSION
The current paper presents the engagement with the workforce programme to overcome language barriers and allow the participants to express them- selves in non-language dependent methods. Current process analysis with cognitive task analysis and interviews have limitations while conducting user engagement across different cultural and language environments. Enga- ging technological solutions and visual props to enable users to articulate their thoughts and start proposing solutions to the existing process challen- ges allows to engage end users in language independent co-creation. The result of such engagement is anticipated to be seen in increased engage- ment and acceptance level of the new process, greater technology readiness, and organizational commitment. The paper serves as protocol for propo- sed user engagement activities and will be delivered across two EC projects (CONVERGING and AI-PRISM) across nine use cases.
ACKNOWLEDGMENT
The work has been conducted on AI-PRISM and CONVERGING projects (101058589 and 101058521, respectively) and the authors would like to thank all consortium members for collaboration.
REFERENCES
Borghouts, J., Eikey, E., Mark, G., De Leon, C., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021). Barriers to and Facilitators of User Engagement With Digital Mental Health Interven- tions: Systematic Review. Journal of Medical Internet Research, 23(3), e24387. https://doi.org/10.2196/24387 Cameron, D., & Collins, E. C. (2021). User, Robot, Deployer: A New Model for
Measuring Trust in HRI. https://doi.org/10.48550/ARXIV.2109.00861 Cechetti, N. P., Bellei, E. A., Biduski, D., Rodriguez, J. P. M., Roman, M. K., & De Marchi, A. C. B. (2019). Developing and implementing a gamification
method to improve user engagement: A case study with an m-Health appli- cation for hypertension monitoring. Telematics and Informatics, 41, 126–138. https://doi.org/10.1016/j.tele.2019.04.007 de Saille, S., Greenwood, A., Law, J., Ball, M., Levine, M., Vallejos, E. P., Ritchie, C., & Cameron, D. (2022). Using LEGO® SERIOUS® Play with stakeholders for RRI. Journal of Responsible Technology, 12, 100055. https://doi.org/10.1016/j.jr t.2022.100055 Eimontaite, I. (2022). Human–Robot Collaboration Using Visual Cues for Commu-
nication. In M. I. Aldinhas Ferreira & S. R. Fletcher (Eds.), The 21st Century Indu- strial Robot: When Tools Become Collaborators (Vol. 81, pp. 71–85). Springer International Publishing. https://doi.org/10.1007/978-3-030-78513-0_5 Eimontaite, I., Cameron, D., Rolph, J., Mokaram, S., Aitken, J. M., Gwilt, I., & Law, J. (2022). Dynamic Graphical Instructions Result in Improved Attitudes and Decreased Task Completion Time in Human–Robot Co-Working: An Experimen- tal Manufacturing Study. Sustainability, 14(6), 3289. https://doi.org/10.3390/su 14063289 Gualtieri, L., Rauch, E., Vidoni, R., & Matt, D. T. (2020). Safety, Ergono-
mics and Efficiency in Human-Robot Collaborative Assembly: Design Guidelines and Requirements. Procedia CIRP, 91, 367–372. https://doi.org/10.1016/j.procir .2020.02.188 Gwilt, I., Rolph, J., Eimontaite, I., Cameron, D., Aitken, J., Mokaram, S., & Law, J.
(2018). Cobotics: Developing a visual language for human-robotic collaborations. Johnson, T. L., Fletcher, S. R., Baker, W., & Charles, R. L. (2019). How and why we need to capture tacit knowledge in manufacturing: Case studies of visual inspection. Applied Ergonomics, 74, 1–9. https://doi.org/10.1016/j.apergo.2018 .07.016 Nakamura, J., & Nagayoshi, S. (2019). The pottery skills and tacit knowledge of
a maser: An analysis using eye-tracking data. Procedia Computer Science, 159, 1680–1687. https://doi.org/10.1016/j.procs.2019.09.338 Pais, S., Petrova, K., & Parry, D. (2021). Enhancing System Acceptance through User- Centred Design: Integrating Patient Generated Wellness Data. Sensors, 22(1), 45. https://doi.org/10.3390/s22010045 Postlethwaite, S., Thiel, K., Fletcher, S., & Eimontaite, I. (2022). A human centric
research of skills and decision making capacity. https://aspect.ac.uk/a-human-cen tric-research-of-skills-and-decision-making-capacity/ Shore, L., Power, V., De Eyto, A., & O’Sullivan, L. W. (2018). Technology Acceptance and User-Centred Design of Assistive Exoskeletons for Older Adults: A Commentary. Robotics, 7(1), 3. https://doi.org/10.3390/robotics7010003 Takamido, R., Kurihara, S., Umeda, Y., Asama, H., Kasahara, S., Tanaka, Y., Fuku-
moto, S., Kato, T., Korenaga, M., Hoshi, M., & Ota, J. (2022). Evaluation of expert skills in refinery patrol inspection: Visual attention and head positioning behavior. Heliyon, 8(12), e12117. https://doi.org/10.1016/j.heliyon.2022.e12117 Tan, J. T. C., Duan, F., Kato, R., & Arai, T. (2010). Safety Strategy for Human–Robot Collaboration: Design and Development in Cellular Manufacturing. Advanced Robotics, 24(5–6), 839–860. https://doi.org/10.1163/016918610X493633 Tofel-Grehl, C., & Feldon, D. F. (2013). Cognitive Task Analysis–Based Training: A
Meta-Analysis of Studies. Journal of Cognitive Engineering and Decision Making, 7(3), 293–304. https://doi.org/10.1177/1555343412474821
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Aspects of Advanced Manufacturing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-56-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional