Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

AzKCLI: a semi-automatic tool for Compositive Lifting Index (CLI) evaluation through Azure Kinect

1 Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Via Giuseppe Campi 287, 41125 – Modena, Italy ↗
2 Department of Sciences and Methods for Engineering, University of Modena and Reggio Emilia, Via Amendola 2, 42122– Reggio Emilia, Italy ↗
Publication year
Digital Object Identifier10.54941/ahfe1005177

Abstract

In modern production systems, prioritizing the safety and well-being of human operator is crucial. Industry 5.0 responds to this need by giving significant importance to the Human Factor (HF) and ergonomics. Our work introduces a semi-automatic tool for Compositive Lifting Index (CLI) calculation for risk detection during multi-task manual lift jobs using the Azure Kinect depth cameras named AzKCLI. We conducted 20 simulations of industrial tasks in our laboratory with a risk assessment from both AzKCLI and expert ergonomic judgment. Additionally, we simulated three tasks taken from the paper that introduced CLI for comparative analysis. Findings reveal a strong agreement between assessments, proposing a novel semi-automatic tool that offers a more objective, economically efficient, and a rapid evaluation of multi-task manual lifting jobs, thus contributing to enhance workplace safety in the Industry 5.0 era.

INTRODUCTION

Ergonomics encompasses the theoretical and foundational comprehension of human behaviour and performance as intentional interactions within sociotechnical systems. It involves applying this understanding to design interactions within real word (Wilson, 2000). In contemporary contexts, ergonomics plays a pivotal role in ensuring safety across various work envi- ronments. Additionally, ergonomic considerations contribute to increased productivity and improved working conditions by replacing traditional meth- ods with new technologies (Fayomi et al., 2021).

The influence of technology is vital for advancing and refining ergonomic techniques within organizations (Canas et al., 2011). The advent of Industry 4.0 introduced new technologies that merged the physical and virtual worlds through cyber-physical systems and interconnected humans, machines, and devices via the Internet of Things. Subsequently, Industry 5.0 reintroduced the essential dimension of a “human/value-centred Industry 4.0”. Unlike

© 2024. Published by AHFE Open Access. All rights reserved. 53

its predecessor, Industry 5.0 is not solely technology-driven but focuses on huma-centricity, ecological considerations, and social benefits. The technolo- gies within Industry 5.0 are integral to system design, aiming to enhance sustainability on bot social and ecological fronts (Müller, 2020).

Specifically, there is a crucial need to concentrate on designing an optimal work environment that prioritize the well-being of operators and emphasizes improving ergonomics to mitigate the occurrence of Work-related Muscu- loskeletal Disorders (WMSDs). Ergonomics plays a key role in achieving this goal. Addressing injuries or disorders related muscles, nerves, tendons, joints, and spinal discs. To reduce the risk level associated with working posture and movements related to manual tasks, it is essential to evaluate risks during pre- production phase when workplaces are designed (Wang et al., 2023). Various observational methods are employed for ergonomic risk assessment based on different activities and types of physical effort. Widely used methods include Rapid Upper Limb Assessment (RULA) (Mcatamney & Corlett, 1993), Rapid Entire Body Assessment (REBA) (Hignett & Ergonomist, 2000), National Institute of Occupational and Safety (NIOSH) Lifting Equation (Waters et al., 1993), Ovako Working posture Analyzing System (OWAS) (Karhu et al., 1977), Occupational Repetitive Action Index (OCRA Index) (Occhipinti, 1998). Contemporary risk evaluation during pre-production phase involves the use of Virtual Reality (VR) and Motion Capture (MoCap) systems. Specif- ically, employing a kinematic suit to capture movements of individual body parts is deemed appropriate (Battini et al., 2018). Collecting data through direct physiological measurements, such as goniometers, force sen- sors, accelerometers, electromyography, and optical markers, provides a high level of accuracy and more objective information (Seo et al., 2014). Joint angles and body posture can be obtained not only through direct measure- ments but also through indirect methods such as Kinect range cameras and computer vision-based approaches (Kaˇcerová et al., 2022).

In the work environment, numerous lifting tasks involve a variety of lifting activities. If detailed information is required for engineering modifications, the multiple-task approach becomes necessary. However, analysing multi- task manual lifting jobs poses greater challenges as each task need to be assessed individually (Waters et al., 2021).

Azure Kinect depth camera was already used for ergonomic scope: (Coruzzolo et al., 2022) proposed an automatic calculation of RULA through the depth camera, comparing results with those obtained through RGB-Based machine vision algorithm; (Lolli et al., 2022) used Azure Kinect to evaluate the ergonomic risk reduction with a height-adjustable mesh truck for picking activities.

Our contribution in this study involves the creation of a semi-automatic tool designed for assessing ergonomic risk associated with multi-task man- ual lifting jobs. Named AzKCLI, our tool utilizes the Composite Lifting Index (CLI) (Waters et al., 1993) combined with a depth camera for data acquisition. Specifically, we employed the Microsoft Azure Kinect for data acquisition and the automatic detection of human body joints. Subsequently,

AzKCLI: A Semi-Automatic Tool for Compositive Lifting Index (CLI) Evaluation 55

our tool computes angles and vectors to derive the CLI and assess the corresponding risk level.

To the best of our knowledge, our tool represents the initial application for the semi-automatic calculation of the CLI for the risk assessment of multi- task manual lifting. We conducted tests using 62 acquisitions with different lifting routines. Each acquisition underwent evaluation by AzKCLI and an ergonomic expert to validate the effectiveness of our novel tool.

The paper is organized as follows: Section 1 details the procedure for conducting the semi-automatic CLI evaluation with AzKCLI while Section 2 outlines the experiment setup, Section 3 presents our results, and Section 4 discusses the conclusion and outlines future research directions.

AZKCLI The NIOSH Lifting Equation (Waters et al., 1993) is suitable for analysing the ergonomic risk associated with a single-task manual lifting job, characterized by consistent task variables throughout the job. In such instances, the impact of tasks on strength, localized muscle fatigue, or whole-body fatigue remains consistent throughout the shift. Conversely, multi-task manual lifting jobs are defined by substantial differences in task variables. In these cases, the NIOSH Lifting Equation proves to be overly restrictive, and the Composite Liftin Index is a more suitable alternative (Waters et al., 2021).

The first guidelines for analysing the physical demands of multi-task manual lifting jobs were introduced in (NIOSH, 1981), incorporating the assessment of the combined effects of all tasks. The novel approach, pre- sented by (Waters et al., 1994), relies on the Composite Lifting Index (CLI), where the cumulative demands of the job are calculated as the sum of the largest Single Task Lifting Index (STLI) and the incremental increases in the CLI as each subsequent task is incorporated. The incremental CLI increase for a specific task is determined as the difference between the Lifting Index (LI) for that task at the cumulative frequency and the LI for that task as its actual frequency. This is done exploiting STLI of each task obtained dividing the average load weight for that task by the respective Single-Task Recom- mended Weight Limit (STRWL) and the Frequency-Independent Lifting Index (FILI) obtained for each task dividing the maximum load weight for that task by the respective Frequency Independent Weight limit (FIRWL). Given STLI and FILI of each task CLI can be calculated as shown in (1) and (2) (Waters et al., 1994).

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

− 1 FM1

)) + (FILI3 ∗( 1 FM1,2,3

− 1 FM1,2 )) . . . . . . . . . .

+ (FILI4 ∗( 1 FM1,2,3,4

− 1 FM1,2,32

)) (2)

+ (FILIn ∗( 1 FM1,2,3, 4..n − 1 FM1,2,3,...,n−1

We developed AzKCLI using Python 3.8, creating an application that semi-automatically computes the CLI by leveraging the Azure Kinect Body Tracking SDK (Microsoft, 2021). An algorithm relying on Convolutional Neural Network (CNNs) is incorporated into the depth camera to identify 32 joints in the 3D space. Each acquisition undergoes pre-processing to con- vert the camera’s output format to the input format compatible with our tool. Then, there is a data cleaning process to remove incomplete or empty infor- mation. Subsequently, our tool can plot the skeleton in the space for each frame in the input file, as shown in Figure 1.

Figure 1: Graphical representation of skeleton from Azure Kinect body hierarchy.
Original PDF Figure 1:

To compute LI and STLI for each task, various geometric functions and additional elements such as points, vectors, and planes were developed. During this phase, we referenced to AzKNIOSH (Lolli et al., 2022) as the calculation of multipliers was identical. The functions involved include:

  • Determining the midpoint between two three-dimensional points and providing the coordinates of the midpoint.
  • Calculating the three-dimensional Euclidean distance between two points.
  • Retrieving coefficients for a plane that passes through a specified point and has a given vector as the normal vector.
  • Computing the coordinates of a point on a specific vector, given the value of one of the coordinates.
  • Calculating the angle in degrees between two three-dimensional vectors.

In order to calculate AzKCLI, certain information needs to be manually entered as input:

  • Number of tasks carried out in the specific job.
  • For each task, the frame at which the task starts.
  • For each task, the frame at which the task ends.
  • Weight of the lifted load in kilograms for each task.
  • Lifting frequency for each task, measured in liftings per minute.
  • Duration of each task.
  • Grip assessment categorized as “good”, “fair,” or “poor.”

AzKCLI: A Semi-Automatic Tool for Compositive Lifting Index (CLI) Evaluation 57

Initially, the tool calculates the FIRWL (Waters et al., 1994) for each activ- ity, utilizing the activity-specific variables and setting the frequency multiplier to 1. It is worth noting that he FIRWL calculation follows the same proce- dure as the Recommended Weight Limit (RWL) in the case of NIOSH Lifting Equation, with the crucial distinction that the frequency multiplier is fixed at 1. Subsequently, it is possible to calculate STRWL. This involves multiply- ing the corresponding FIRWL by the appropriate frequency multiplier. Then, to obtain FILI for each tasl, the load weight for each activity is divided by the corresponding FIRWL. Similarly, the STLI, is obtained by dividing the load weight for each task by the corresponding STRWL. Once all the neces- sary measures have been calculated, they are utilized in Equation 1 and 2 to obtain a unified measure of the risk level associated with all the tasks per- formed in that job. The AZkCLI output is an Excel file where all the results are saved.

EXPERIMENT SETTING To validate AzKCLI, we conducted simulations of an industrial environment in laboratory setting, replicating various multi-task manual lifting scenar- ios performed by two volunteers. The aim of the experiment was to ensure heterogeneity in the validation process. The experimental setup included an Azure Kinect for recording, a PC Alienware, an industrial table measuring 150x80x90 (h) cm, a shelving unit sized 2000x500x1200 (h) cm for loading and unloading boxes from different shelves (illustrated in Figure 2), an indus- trial pallet 120x80 cm, an industrial mesh truck 120x100x80(h) cm, one container 60x40 cm weighted 0.5 kg and different lifted boxes with following characteristics:

  • Three boxes sized 35x22x13 (h) cm, weighted respectively 2, 3 and 5 kg.
  • One box sixed 15x15x10(h) cm, weighted 1.5 kg.
  • Five boxes sized 60x40x40(h) cm, each one weighted 12 kg.
Figure 2: Graphical representation of the shelving used in the experiment.
Original PDF Figure 2:

62 acquisitions were analysed both by an ergonomist expert and AzKCLI. The acquisitions represented different industrial routines that an operator

performs to move different loads from an origin destination to the final loca- tion. The start and the end frame of each activity, considered in the manual assessment, coincide with those taken into account to run AzKCLI. To ensure a more accurate evaluation and reliable measurements, the ergonomist was provided with all relevant information regarding the handled load, support planes for the loads, and the characteristics of the monitored subject. Once the necessary measurements for calculating the multipliers were obtained, the professional utilized conversion tables (Waters et al., 1994), distinguishing their approach from the automatic tool, which relies on specific equations.

To replicate industrial activity, the tasks were structured as follows: volun- teer lifted boxes from various levels. Subsequently, they scanned the code on each box and positioned it on either the mesh truck or the pallet, as instructed. Some boxes are designated to be put in a container before the destination placement. Figure 3 illustrated a depiction of the tasks.

Figure 3: Representation of analysed performed tasks.
Original PDF Figure 3:

Manual inputs were entered both in AzKCLI and provided to the expert, encompassing:

  • The number of tasks performed in the specific job, which varied for each acquisition.
  • For each task, the frame indicating the task’s start.
  • For each task, the frame marking the task’s end.
  • The weight of the lifted load for each task, set at 1 kg per box.
  • Lifting frequency for each task, measured in liftings per minute, with each lift executed one a minute.
  • A job duration of 8 hours.
  • A grip assessment categorized as “good”.
  • Volunteer is male falling within an age range of 18 to 45 years, resulting in a set load constant of 25 kg.

AzKCLI: A Semi-Automatic Tool for Compositive Lifting Index (CLI) Evaluation 59

RESULTS

The 62 videos captured by Azure Kinect have been converting from .mkv files to.json files, to give as input for the semi-automatic Composite Lifting Index carried out with AzKCLI. While, the expert ergonomist, analysed the .mkv with observational method. Results are shown in Figure 4.

Figure 4: Graphical representation of results comparing AzKCLI with ergonomic expert evaluation.
Original PDF Figure 4:

Statistical analysis was done to demonstrate the tool reliability. The mean difference between CLI calculated by AzKCLI and ergonomist judgement is 0.058 with a standard deviation of 0.156. Specifically, the found agreement between the two methods is 77%. Finally, we discretized the two CLI in four categories to apply linear weighted Cohen’s Kappa coefficient of agreement (Fleiss et al., 1969). The Cohen’s Kappa coefficient results is a value of 0.81 that corresponds to an almost perfect agreement using the Landis and Koch Scale (Landis & Koch, 1977). Statistical analysis shown that AzKCLI is a reli- able option to evaluate the ergonomic risk associated with multi-task manual lifting jobs.

CONCLUSION

In this work, a new semi-automatic evaluation of CLI was presented. Through the detection of 32 body joints position provided by the depth cam- era Azure Kinect, required measurements were determined to calculate the final CLI. We compared AzKCLI with the evaluation of an ergonomic expert to demonstrate the tool reliability. Specifically, following conclusions were drawn:

  • Almost perfect agreement was highlighted between AzKCLI’s results and ergonomic expert’s assessment.
  • The usage of AzKCLI enables to punctually detect distances used for CLI calculations.
  • Some AzKCLI’s inputs cannot be automatically detected, and it could be useful a neural network to completely automatize the process.
  • Major limitation of AzKCLI is due to the depth camera. It is an optical sensor and unavoidability occlusions risk compromising data collection. In order to avoid this problem, future research could include data collected with multiple Kinect.

Extensions of this work can include both the comparison of results with other methods for ergonomic risk assessment and the integration of a neu- ral network to automatically determined number of liftings, frequency and duration of the task, beginning and end of the lift.

REFERENCES

Battini, D., Calzavara, M., Persona, A., Sgarbossa, F., Visentin, V., & Zennaro, I. (2018). Integrating mocap system and immersive reality for efficient human- centred workstation design. 51(11), 188–193. https://doi.org/10.1016/j.ifacol .2018.08.256 Canas, J. J., Velichkovsky, B. B., & Velichkovsky, B. M. (2011). Human Factors and

Ergonomics. In IAAP handbook of applied psychology (pp. 316–338). Coruzzolo, A. M., Lolli, F., Amicosante, N., Kumar, H., Thupaki, P., & Agarwal, S. (2022). Comparing semiautomatic Rapid Upper Limb Assessments (RULA): Azure Kinect versus RGB-based machine vision algorithm. Physical Ergonomics and Human Factors, 63. https://doi.org/10.54941/ahfe1002596 Fayomi, O. S. I., Akande, I. G., Essien, V., Asaolu, A., & Esse, U. C. (2021). Advances

in Concepts of Ergonomics with Recent Industrial Revolution. IOP Conference Series: Materials Science and Engineering, 1107(1), 012010. https://doi.org/10. 1088/1757-899x/1107/1/012010 Fleiss, J. L., Cohen, J., & Everitt, B. S. (1969). Large Sample Standard Errors of Kappa and Weighted Kappa 1. In Bulletin (Vol. 72, Issue 5). Hignett, S., & Ergonomist, L. M. (2000). Rapid Entire Body Assessment (REBA). In

Applied Ergonomics (Vol. 31). Kaˇcerová, I., Kubr, J., Hoˇrejší, P., & Kleinová, J. (2022). Ergonomic Design of a Workplace Using Virtual Reality and a Motion Capture Suit. Applied Sciences (Switzerland), 12(4). https://doi.org/10.3390/app12042150 Karhu, O., Kansi, P., & Kuorinka, I. (1977). Correcting working postures in industry:

A practical method for analysis. In ApplledErgonomtcs (Vol. 8). Landis, J. R., & Koch, G. G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1), 159. https://doi.org/10.2307/2529310 Lolli, F., Coruzzolo, A. M., Forgione, C., Gonçalves, P., & Neto, T. (2022).

Ergonomic risk reduction: a height-adjustable mesh truck for picking activities evaluated with a depth camera. AHFE, 2023, 2023. https://doi.org/10.54941/ah feXXXX Mcatamney, L., & Corlett, E. N. (1993). RULA: A survey method for the investigation of work-related upper limb disorders. In Applied Ergonomics (Vol. 24, Issue 2). Microsoft. (2021). About Azure Kinect DK. https://learn.microsoft.com/en-gb/azur

e/kinect-dk/about-azure-kinect-dk Müller, Julian. (2020). Enabling Technologies for Industry 5.0 : results of a workshop with Europe’s technology leaders.

AzKCLI: A Semi-Automatic Tool for Compositive Lifting Index (CLI) Evaluation 61

NIOSH. (1981). Work Practices Guide for Manual Lifting. NIOSH Technical Report No. 81–122. Occhipinti, E. (1998). OCRA: a concise index for the assessment of exposure to

repetitive movements of the upper limbs. Ergonomics, 41(9), 1290–1311. https: //doi.org/10.1080/001401398186315 Seo, J., Starbuck, R., Han, S., Lee, S., & Armstrong, T. (2014). Motion Data- Driven Biomechanical Analysis during Construction Tasks on Sites. Journal of Computing in Civil Engineering, 29, B4014005. https://doi.org/10.1061/(ASCE) CP.1943–5487.0000400 Wang, J., Li, X., Han, S., & Al-Hussein, M. (2023). 3D standard motion

time-based ergonomic risk analysis for workplace design in modular construc- tion. Automation in Construction, 147, 104738. https://doi.org/10.1016/j.autcon .2022.104738 Waters, T. R., Putz-Anderson, V., & Garg, A. (1994). Applications manual for the revised NIOSH Lifting Equation. Waters, T. R., Putz-Anderson, V., Garg, A., & Fine, L. J. (1993). Revised NIOSH

equation for the design and evaluation of manual lifting tasks. Ergonomics, 36(7), 749–776. https://doi.org/10.1080/00140139308967940 Waters Thomas R, Putz-Anderson, V., & Garg, A. (2021). Applications Manual for the Revised NIOSH Lifting Equation. Wilson, J. R. (2000). Fundamentals of ergonomics in theory and practice.

Cover of the proceedings volume Physical Ergonomics and Human Factors

Explore the complete digital volume

Physical Ergonomics and Human Factors

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-23-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →