Kartikeya Walia, Philip Breedon (2024). Assembly Complexity Index (ACI): A Framework to Evaluate Assembly Process for Validating a Modular Robotic Design. In: Human Factors in Design, Engineering, and Computing. AHFE Open Access, vol. 159, pp. 761–771. https://doi.org/10.54941/ahfe1005643
The assembly of equipment necessitates varying degrees of expertise, with complexity often escalating alongside technological advancements. While automation has reduced the workload in manufacturing and assembly lines, repair and maintenance still require a significant user skillset. This research focused on developing a modular robotic system with straightforward assembly and disassembly, requiring minimal robotics expertise from end users. A modular robotic system offers benefits such as shorter repair times leading to reduced downtimes on a factory shop floor, options for task-agnostic reconfiguration and deployment, and potential reductions in initial investment costs.To validate this hypothesis, a study was conducted with twelve participants with differing expertise in tools, hardware, and construction. Direct evaluation of personal and workplace attributes such as workload, task complexity, prior expertise and learning is often indiscernible and non-comparable. Thus, it was essential to establish a tangible workflow to evaluate and monitor the design's effectiveness and any modifications' impact on assembly ease. The study employed the Task Complexity Index (TCI) and NASA Task Load Index (TLX) adapted to measure task complexity and user workload. Both TCI and TLX have been used independently in various studies and a correlation between the two was identified. Combining data on task complexity and workload provided a comprehensive evaluation of the assembly process.Results indicated a marked improvement in the Assembly Complexity Index (ACI) during the second phase of experiments due to participant learning and a lower time (p = 0.026) required for completion of a much more complicated task demanding a higher workload (p = 0.014). This research aims to establish a framework for identifying an Assembly Complexity Index (ACI) using these the subjective workload and complexity assessment tools. The study considered factors such as the number of components, operations, and tools required. In addition, it acknowledged that factors like the availability of resources, component size and weight, operation complexity, and tool availability also impact the overall assembly complexity.
With the increasing adoption of robotics and automation in industry, it has become evident that future factories will require some form of robotics. Although many robotic solutions are available off-the-shelf, there remains a clear hesitance in certain industries with lower payload capacities. The Food and Beverage sector (Müller, 2023) contributed to only 3% of global annual installations of industrial robots and showed no increase in the number of units installed compared to the previous year. As most available manipulators are designed for heavy-duty factory tasks, making them unsuitable for applications requiring lightweight payloads. Furthermore, off-the-shelf robots present additional challenges such as vendor lock-in (Markl et al., 2021) and dependence on the original equipment manufacturer (OEM) for repairs and maintenance, which often result in significant downtimes and adversely affect throughput (Bard, 1986). These issues are intensified when there is a need for task changes on the factory shop floor. Fixed degrees-of-freedom (DoF), specific manipulator specifications like payload capacity, reach, and work envelope necessitate considerable investment costs for re-deployment to justify any task changes, including the decommissioning of older robotic solutions, managing their end-of-life, and purchasing and deploying newer, suitable robots or modifying and re- designing the task to fit within the manipulator limitations (Chen and Yim, 2016). Task-agnostic manipulator solutions are therefore essential. Research into task-agnostic robotic solutions has been ongoing since the early days of industrial robotics, leading to many modular robotic theories in the literature. Some of the most cited research developments in modular robotics include the Martonair Modular System at the Loughborough University of Technology (now known as Loughborough University) (Harrison et al., 1986), the Modular Robot System at the University of Stuttgart (Wurst, 1986), the Reconfigurable Modular Manipulator System at the Carnegie Mellon University (Schmitz et al., 1988), The Dynamically Reconfigurable Robotic System at the Science University of Tokyo (Fukuga and Nakagawa, 1988), the Structural Modules at the University of Texas (Tesar and Butler, 1989), the Rotary Joint based remote actuation at the University of Toronto (Benhabib and Dai, 1991) and the TOshiba Modular Manipulator System, TOMMS (Matsumaru, 1995). These developments propose various solutions for module/unit-based designs and modularity in robotic systems, offering inventories of components that can be reconfigured into desired manipulator geometries. However, most suggested actuator units are too heavy for practical industrial tasks, mainly due to the unavailability of state-of-the- art, lightweight electric motors during the late 20th century. Modularity and reconfigurability seem to be obvious design solutions for bespoke applications. The objective of ongoing research in the area of “mechanical design of modular robots” was to develop an inventory of basic modular units with modern off-the-shelf electronics and a supported robotic software system interface.
Assembly Complexity Index (ACI): A Framework to Evaluate Assembly Process 763
The research focused on developing a modular manipulator to provide bespoke solutions for end-users’ changing task needs. A design-to- fit approach was employed during the development stages, utilising a combination of Generative Design (Walia et al., 2021) and Additive Manufacturing (Walia et al., 2021) to optimise both the design and manufacturing processes respectively, and ensure lightweight structures. The primary aim was to ensure the developed robotic system remained throughout its lifecycle, especially for end-users with low technical expertise. Key features included ease of assembly, integration, deployment, operation, repair, decommissioning, and reusability to enhance system accessibility.
One essential requirement in robotics, and automation systems in general, is managing the complexity involved and ensuring the correct level of expertise for the assembly and deployment of systems. While lowering system complexity is not always feasible, evaluating subjective complexity is crucial for ensuring proper training of the end-user. An inverted U-shaped relationship typically correlates task demand with performance, where performance decreases as workload increases and resources become limited (Lysaght et al., 1989).
The presented study of mental workload and subjective task complexity was conducted through collaboration between NTU and PepsiCo. Task complexity was assessed by volunteers rating their experience with the task using a developed Task Complexity Index (TCI) scale, while subjective mental workload was measured using the NASA Task Load Index (NASA-TLX) (Hart and Staveland, 1988). A critical consideration throughout the research was ensuring a lower system’s assembly complexity. This drove the design iteration phases, helping to improve the design and make informed, data- driven decisions. The developed ‘Assembly Complexity Index’ (ACI) scale was used to evaluate this aspect. This paper presents the methodology developed to utilise the TLX and TCI indicators to collect data for the corresponding subjective sub-scales, followed by normalisation the data, and combining it with or without a weight function to derive an Assembly Complexity Index.
To validate the developed ACI framework, a study was conducted with volunteers from PepsiCo and the NTU Engineering Department. Participants followed a comprehensive Assembly Manual and provided valuable feedback for design iterations and improvements. The ACI framework quantitatively considered a combination of both workload and task and demonstrated the impact of supervised and unsupervised assembly (with and without an assembly manual, and with and without prior experience) on the reported system complexity and the time taken to complete the assembly.
METHODOLOGY
It was crucial to model an appropriate workflow to tangibly evaluate and monitor the effectiveness of the design and any design iterations/modi- fications on the ease of assembly. This was used to quantitatively determine and assign an index for the robotic assembly complexity. Task Complexity Index (TCI) and NASA Task Load Index (TLX) scales were appropriately
764 Walia and Breedon
modified and used to measure the task complexity and user workload, respectively. The section goes through the different stages of the methodology covering briefly the modified NASA TLX and TCI indicators and how these have been utilised to extract data and further combined to develop a universal ACI.
NASA TLX and TCI Indicators The NASA Task Load Index (NASA-TLX) is a widely used, subjective workload assessment tool that evaluates an individual’s perceived workload across six indicators: Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration. This study utilised the NASA-TLX to measure the mental workload of participants during the assembly of the modular robotic system.
Figure 1: NASA task load indeX scale.
A 21-point rating scale (see Figure 1) was used for each of the six components. Equation 1 was used for calculating the TLX score (am).
Pn
Equations and mathematical formulas are available in the original paper PDF.
Assembly Complexity Index (ACI): A Framework to Evaluate Assembly Process 765 y = score for the nth indicator by the jth participant wj = weight from the subscale
A task complexity rating questionnaire was developed for this study. This was based on a previous studies where after completing simulated scenarios, operators rated the contribution of each item to the difficulties in solving tasks. Factor analysis of the ratings identified eight interpretable factors (Braarud, 1998 and Collier, 1998). Task Complexity Index (TCI) relates to the various variables to address difficulties in solving a task: Root cause difficulties, the spread of information, ambiguous information, coordination, guidance information, attention demand, severity for plant safety, and temporal demand. and combining it with or without a weight function to derive an Assembly Complexity Index.
Adding a 7-point Likert scale to the description of each factor developed a TCI scale (see Figure 2). Equation 3 was used for evaluating the TCI (bn). This involved using the linearisation of the area ratio obtained from the octagonal radar chart (see Figure 3).
Figure 2: Developed task complexity index scale.
Equations and mathematical formulas are available in the original paper PDF.
4
v u u u t
Equations and mathematical formulas are available in the original paper PDF.
72 2 sin π bn= nth indicator index n = number of subscales m = participant number x = 8 - score for the nth indicator by the ith participant
766 Walia and Breedon
Figure 3: Task complexity index indicators on a radar chart.
Normalisation and Assembly Complexity Index This was followed by the normalisation of the data for both TLX and TCI to a 5-point scale using equations 4 and 5.
a′
am= mth TLX bm= mth TCI m = participant number maximum weighted TLX value = 18.67 maximum TCI = 7 Assembly Complexity Index (ACI, cm) was calculated using the equation 6.
m wa + b′
m wb 2 (5)
a
Assembly Complexity Index (ACI): A Framework to Evaluate Assembly Process 767
m = participant number The weights wa and wb (between 0 - 1) calculated by averaging an importance score between the workload and the complexity (see Figure 4) and normalised to 1, such that wa + wb = 1.
Figure 4: Weights for TLX and TCI normalised to 1.
Figure available in the original PDF paper.
Experimental Study
Figure 5: Some participants performing tasks during the experimental study.
The study conducted involved 12 participants (see Figure 4) visiting twice for 2-hour long sessions with a gap of two weeks between the two sessions. The participants were provided no prior information about the details of the assembly tasks. During the first visit (Task 1), participants were allotted a maximum time of 90 minutes to get familiar with the provided assembly manual for the robotic joint sub-assemblies and utilise the provided 3D printed parts, electronic components and tools to complete the assigned sub- assembly of a single robotic manipulator joint. The completion of the task involving the electro-mechanical assembly culminated with the control of the actuated robotic joint using a simple GUI based controller.
The second visit (Task 2) involved the participants to assemble a 2 DoF robotic manipulator configuration without the use of the assembly manual using the given tools, 3D printed parts and electronic components but without a time limit.
A completion time for each participant was also recorded for both the tasks. At the end of both tasks the participants were required to fill out the
768 Walia and Breedon
NASA-TLX and developed TCI questionnaires. The completion times and the calculated TLX, TCI and ACI values are presented in the Table 1.
Table 1. Recorded completion times and calculated TLX, TCI and ACI values.
This was followed by a statistical analysis to validate the hypothesis and extract correlations between several variables. This process particularly proved to be useful for determining the specific areas of improvement for future design iterations and task related resources.
Reliability of the Recorded TCI and TLX To evaluate the reliability of the recorded data, the internal consistency was estimated using the Cronbach’s alpha. A low to moderate alpha of 0.707 (> 0.7) and 0.724 (> 0.7) was calculated for the task complexity (n = 8) during the task 1 and 2, respectively. This was an acceptable internal consistency for the TCI rating scale (Murphy and Davidshofer, 1994).
In comparison, Cronbach’s alpha for the NASA TLX was .831 and .777 for Task 1 and Task 2, respectively.
Correlation A Pearson correlation coefficient (Cohen et al., 2009) was calculated for the measured variables with time and is presented in Table 2.
A negative correlation between TCI and Time for Task 2 demonstrates that a lower time was required for a task which had subjectively higher complexity in comparison to Task 1 due to no availability of the assembly manual and a bigger assembly to be completed. This could be due to a combined effect of the learning experience from Task 1 and no time limit for Task 2.
A higher workload has been associated with a higher time taken for the assembly completion though-out the study.
Assembly Complexity Index (ACI): A Framework to Evaluate Assembly Process 769
Descriptor Correlation Coefficient (r) TCI and Time (task 1) 0.134 TLX and Time (task 1) 0.398 ACI and Time (task 1) 0.404 TCI and Time (task 2) −0.223 TLX and Time (task 2) 0.274 ACI and Time (task 2) 0.238 TCI and Time (overall) 0.071 TLX and Time (overall) 0.280 ACI and Time (overall) 0.275
Correlation between TCI and TLX for both tasks were positive (0.158 and 0.181, respectively) clearly indicating an increase in workload with the increasing complexity of the task.
Paired t-Tests One-tailed paired t-tests were conducted for the data comparing the two tasks undertaken.
The evaluated TCI did not show a significant difference in the two tasks, consolidating the potential effects of learning and experience and the similar assembly steps required for the developed manipulator. This also signified the ease of assembly and a relatively lower technical-expertise requirement from the end-user.
Workload reported was significantly higher in task 2 in comparison to task 1 (p = 0.014). The overall Assembly Complexity Index increased for the task 2 (p = 0.046).
The p-value of 0.026 (< 0.05) indicated a significantly lower time required to complete the assemblies on the second attempt, additionally confirming the positive impact on the assembly efficiency of the end-used due to a previous experience.
DISCUSSION AND CONCLUSION In addition to the quantitative data and presented analysis, the qualitative participant feedback at the end of the study aligned with the observed effects of learning and ease of assembly during the second attempt. Although, the current methodology does not include a tangible method to directly measure the effects of learning and experience of a participant, but the completion of Task 2 without the availability of the Assembly Manual and a significantly lower time required even with a higher workload indicates the correlation with the previous Task and the similarity of assembly steps.
The research project involved the development of an easy-to-assemble modular manipulator to suit the bespoke and changing needs of the industry shop-floor. The presented framework to evaluate the Assemble Complexity Index of an assembly task of a robotic manipulator can be universally applied to most tasks under consideration either directly or after appropriate
770 Walia and Breedon
modifications. In the research project the ACI methodology proved to be beneficial to improve the design of the manipulator with considerations. Observing the raw data for the specific indicators revealed the potential for improvements in the Assembly Manual provided (TCI →Ambiguous Information and Spread of Information).
The presented ACI methodology can be used as an evaluation tool at the initial design and development stages to make informed and data- driven design decisions and iterations. The study conducted also showed the potential use of ACI as a monitoring tool for assembly and disassembly stages during deployment, maintenance, repair and decommissioning.
A numerical ACI allows users to train for different levels of complexity effectively and monitor access to specific sub-assembly stages for a simple system, such as the one presented here, or even a highly complex off-the-shelf manipulator robot. The developed methodology is universally applicable to any task involving several stages of complexity. It helps monitor, train, ensure the safety of both the end user and the equipment during assembly, and aids in making informed decisions during the design stages.
ACKNOWLEDGMENT
The authors would like to acknowledge PepsiCo International and Nottingham Trent University for funding the research.
REFERENCES
Bard, J. F. (1986). An assessment of industrial robots: Capabilities, economics, and impacts. Journal of operations Management, 6(2), 99–124. Benhabib, B., & Dai, M. Q. (1991). Mechanical design of a modular robot for
industrial applications. Journal of Manufacturing Systems, 10(4), 297–306. Braarud, P. Ø. (1998). Complexity factors and prediction of crew performance (HWR-521). Halden, Norway: OECD Halden Reactor Project. Chen, IM., Yim, M. (2016). Modular Robots. In: Siciliano, B., Khatib, O. (eds)
Springer Handbook of Robotics. Springer Handbooks. Springer, Cham. https: //doi.org/10.1007/978-3-319-32552-1_22 Cohen, I., Huang, Y., Chen, J., Benesty, J., Benesty, J., Chen, J., ... & Cohen, I. (2009).
Pearson correlation coefficient. Noise reduction in speech processing, 1–4. Collier, S. (1998). Development of a diagnostic complexity questionnaire. Fukuda, T., & Nakagawa, S. (1988). Approach to the dynamically reconfigurable robotic system. Journal of Intelligent and Robotic Systems, 1, 55–72. Harrison, R., Weston, R. H., Moore, P. R., & Thatcher, T. W. (1986). Industrial
application of pneumatic servo-controlled modular robots. In Advances in Manufacturing Technology: Proceedings of the First National Conference on Production Research (pp. 229–236). Springer US. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology (Vol. 52, pp. 139–183). North-Holland. Lysaght, R. J., Hill, S. G., Dick, A. O., Plamondon, B. D., Linton, P. M., Wierwille,
W. W., Zaklad, A. L., Bittner, A. C., & Wherry, R. J. (1989). Operator workload: Comprehensive review and evaluation of operator workload methodologies (Tech. Rep. 851). Fort Bliss, TX: U. S. Army Research Institute, Field Unit.
Assembly Complexity Index (ACI): A Framework to Evaluate Assembly Process 771
Matsumaru, T. (1995, May). Design and control of the modular robot system: TOMMS. In Proceedings of 1995 IEEE international conference on robotics and automation (Vol. 2, pp. 2125–2131). IEEE. Murphy, K. R., & Davidshofer, C. O. (1994). Psychological testing: Principles and
applications. Englewood Cliffs, NJ: Prentice Hall. Markl, E., Bauer, J. M., Riepl, D., & Steiner, T. (2021). Logic control of an industrial automation cell with ros as part of smes and industry 4.0 interconnection. In Digital Conversion on the Way to Industry 4.0: Selected Papers from ISPR2020, September 24–26, 2020 Online-Turkey (pp. 210–221). Springer International Publishing. Müller, C. (2023) World Robotics 2023 – Industrial Robots, IFR Statistical
Department, VDMA Services GmbH, Frankfurt am Main, Germany. Schmitz, D., Khosla, P., & Kanade, T. (1988). The CMU reconfigurable modular manipulator system. Tesar, D., & Butler, M. S. (1989, January). Modularity in robotic systems. In
JPL, California Inst. of Tech., Proceedings of the NASA Conference on Space Telerobotics, Volume 1. Walia, K., Khan, A., & Breedon, P. (2021). Polymer-based additive manufacturing: process optimisation for low-cost industrial robotics manufacture. Polymers, 13(16), 2809. Walia, K., Khan, A., & Breedon, P. (2021). The generative design process for robotic
design applications. Journal of Additive Manufacturing Technologies, 1(2), 528–528. Wurst, K. H. (1986). The conception and construction of a modular robot system.
In Proc. 16th ISIR (Vol. 37).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Design, Engineering, and Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-35-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional