Jussi Okkonen, Mia Laine, Christian Andersson (2024). Technology relationship, algorithmic thinking and task performance with UxV. In: Human Factors in Design, Engineering, and Computing. AHFE Open Access, vol. 159, pp. 1111–1117. https://doi.org/10.54941/ahfe1005678
There is evidence that subjectively assessed technology relationship, sense of self-efficacy, algorithmic thinking and operation with unmanned vehicles are in correlation. The complex skillset required for both operators and UxV operators requires understanding on nature of information, algorithmic, computational, and epistemic thinking. Based on framework of algorithmic thinking this paper connection between algorithmic thinking, technology relationship, and task performance in operation supported with fleet of unmanned land vehicles. The paper is based on extensive quasi-experiment (n=500) in simulation environment. Key results presented are related to connections between different personal attributes, digital literacy and tasks performance with autonomous systems in the battlefield.
Based on study by Okkonen et al. (2024) the role of technology relationship and algorithmic thinking as personal characteristics in functioning with autonomous capabilities such as unmanned vehicles (UxV’s) are the factors forecasting better task performance. The basic proposition is that conscripts with algorithmic thinking above the median are more able to interact with autonomy. The findings point out coexistence of sense of self- efficacy, positive attitude towards technology, digital literacy and capability to algorithmic thinking. The implications of manifestations of digital literacy can be presented on several levels. As discussed in Xu (2020) the interplay with human and system requires understanding on limitations of autonomous system as well as limitations of operator of it. Moreover, operating the UxV’s, as well as being on mission with such capabilities requires human-autonomy teams human trust (Wohleber et al., 2023). Such trust is built on experience on technology, general technology relationship and agency over technology (e.g., Freedy et al., 2007). In several military cases there is requirement for human-in-the-loop or human-on-the-loop, yet the teaming should be based on expected synergy. In this paper the rationale for team building is not in focus, but the focus is more on performance of people with UxV. Rossiter (2020) forecasts unmanned vehicles becoming more and
more common in land operations due rapid technological advances, yet there is need to put attention human team mates as well. The nature of mission sets the limits of utilisation of such capabilities (Matejka, 2020).
Rödel et al. (2014) discuss the user acceptance and user experience of autonomy as a sum of experienced utility of autonomy and perceived user experience built on of ease of use, attitude towards using autonomy, behavioral intention of the system, and trust and fun. In drone (UAV) context Christ et al. (2016) emphasize similar factors, yet trust on technology, in this case the integrated system, gains importance as degree of autonomy increases. Trust can be seen as the flip side of interaction or controlling (cf. Goodrich & Schultz 2007; Crandall et al., 2005). The relationship with autonomy builds up on technology relationship, personality and user experience. In this paper the hypothesis is set on those. In addition, logical-mathematical intelligence, such as algorithmic thinking, should also positively affect cooperation with autonomy. As stated in Okkonen et al. (2024) technology self-efficacy, algorithmic thinking and motivation are connected. Putting above mentioned factors together then interaction is defined by personal attributes emphasizing prior experience on technology and capability to understand the nature of it. However, personality features are also important as stated in Svendsen et al. (2011). This should be taken into account when behavioral intention is addressed as a sum of motivational factors. Motivation is also taken into account when interaction is studied. Behavioral intention becomes visible through existing or non-existing interaction with autonomy.
The complex skillset required for both operators and UxV operators requires understanding on nature of information, algorithmic, computational, and epistemic thinking. Based on framework of algorithmic thinking the aim is to study the connections between algorithmic thinking, technology relationship, and task performance in operation supported with fleet of unmanned land vehicles. Based on the results presented in Okkonen et al. (2024), hypothesis for this study is derived of the characteristics of the individual, i.e., that those who succeed in the test are more able to interact with autonomy. This is also aligned with the finding that positive attitude towards technology in general enhances utilisation of technology in particular.
RESEARCH SETTING AND RESULTS The participants of experiment consisted of 431 conscripts, 27 commissioned officers and 37 armored reserve students all from the armored brigade of Finland. The experiments were run during May and June 2024. Participants were allocated to different roles of defending infantry troops and attacking troops as part of the mechanized infantry, while 5 staff officers controlled simulated infantry troops operating the UGVs. The defending and attacking troops were commanded by senior officers, yet their role is considered neutral. In half of the scenarios UGV’s were operated by human operator with direct communication possibility by squad leaderes. Half of the simulated scenarios UGVs were operated using the wizard of Oz method, where human
Technology Relationship, Algorithmic Thinking and Task Performance With UxV 1113
operators represented the AI capabilities. In latter scenarios squad leader could steer UGV’s indirectly by setting it to execute certain task or setting it to a certain location. A total of 48 battle simulations were fought including 4 control scenarios. R, R Studio (The R foundation, 2024; RStudio Team, 2024) were used for the analysis. Graphics were produced with ggplot2 package (Rosenberg, 2018). The analysis is exploratory and focuses on broad hypotheses regarding the associations between technology relationship, motivational aspects, algorithmic thinking skills and self-assessed performance.
Tehcnology relationship, motivational aspects, and algorithmic thinking were measured by similar pre-tested questionnaire as discussed in Okkonen et al. (2024). Item means were similarly calculated for appropriate constructs. Answers to the graded items were awarded a point only if and only if they were fully correct. The data of 2023 compared to data of 2024 is similar and there is no significant difference. Overall, the responses of 431 conscripts are very similar and all differences between the responses fit well to 5 % error margin. In this sample, the participants’ technology relationship has an association with the outcome of the algorithmic thinking section. Participants with poor technology relationship had a small number of correct answers, with the binned groups’ mean 2 correct answers. Conversely, participants with better relationships with technology answered typically 4– 5 questions correctly, or more as illustrated in Figure 1. Endpoint values of motivational aspects seemed to be associated with decreased scores in the algorithmic thinking skills section as illustrated in Figure 2. The associations of individual composite scores of relationships with technology and motivational aspects fit also the working proposition of this paper.
Figure 1: Technology relationship and algorithmic thinking.Figure 2: Algorithmic thinking scores and motivational factors.
Lowest scores in utility values and performance-approach orientation seemed associated with lower amount of fully correct answers to algorithmic thinking items. Overall, performance-approach orientation could have some association with more correct answers, but the differences in categorized motivational aspect scores were subtle. Scores from a three-factor solution exhibited significant (p<0.05) coefficients on a binomial model predicting the probability of answering a single question correctly. Factors relating to technology relationship items were found to have more statistically significant coefficients as well as larger coefficient values than factor scores of motivational factors. It is noteworthy, that not all coefficients are positive.
Figure 3: 3 factor solution of technology relationship items have significant effects on the probability of answering correctly to individual questions. (The dots denote coefficients and vertical lines 95% confidence interval).
Technology Relationship, Algorithmic Thinking and Task Performance With UxV 1115
As described in Figure 3 the connection between technology relationship and performance in algorithmic thinking test show high likelihood to excel in test and have positive technology relationship. The causal relationship is not clear but there is significant co-existence.
Figure 4: 3 factor solution of motivation factor items have significant effects on the probability of answering correctly to individual questions. (The dots denote coefficients and vertical lines 95% confidence interval).Figure 5: Coefficients of an ordinal logistic regression model explaining the probability of having a consecutively higher performance score. (The dots denote the estimated coefficient, and the horizontal lines mark the 95% confidence interval).
The association between self-assessed performance and the selected items was assessed with an ordinal logistic regression model (cf. Figures 4 and 5). This method allows for estimating the odds of being at or above each response
category. As the self-assessment was done one five-point diverging scale, most responses were neutral. Therefore, the extreme scores were combined to create three categories, and oversampling was used to even out remaining differences in category sizes. A stepwise algorithm was implemented, which iteratively refines the model minimize the Akaike Information Criterion (AIC). Scenarios, use of UGVs and Algorithmic Thinking Score would have been dropped by the algorithm, but they were chosen to be included for other reasons. The model had an accuracy of 41.5%. Scenarios were included in the model control for the scenario environment and for the effect of having switched sides between scenarios 2 and 3. Scenarios 2 and 4 also had autonomous UGVs, whereas in scenarios 1 and 3 they were remotely operated. As seen in Figure 6, having used UGVs seemed to have a positive effect on the self-assessed performance. Out of the selected items, performance-approach orientation seemed to be associated with a lower performance score, whereas difficulty in use of technology was associated with a higher performance score. Algorithmic thinking score had a very small, non-significant effect. Nevertheless, there an effect could still exist with objective task-performance measures as illustrated in Figure 6.
Figure 6: Driving factors and self-assessed performance.
DISCUSSION
Based on the exploratory results detailed in this paper, a positive relationship with technology seemed to have an association with algorithmic thinking skills. We propose that a compounding effect of this relationship and higher algorithmic thinking skills could have an effect task performance with UxVs. Self-assessed performance evaluation score was used in this paper as the metric of task performance. However, an extensive analysis is necessary to take into consideration the accuracy of this assessment, as well as weather the overall performance of the group affects the individual assessment. The analysis in this paper does also not take into account for the participants
Technology Relationship, Algorithmic Thinking and Task Performance With UxV 1117
motivation during the experiment. Motivation is likely to interfere with task performance, using new technology as well as the self-assessment. The presented results implicate the connection between task performance and personal attributes. These findings should be utilised when allocating people to UGV users and operators. By the findings it is possible to define set of criteria for choosing people to those tasks. Also, but not as evidently, exclusion criteria could be defined too.
ACKNOWLEDGMENT
The study for this paper was funded by the grant of PVTO program. Authors acknowledge the contribution of the participants, as well as the other people in Finnish Defence Forces.
REFERENCES
Christ, P. F., Lachner, F., Hösl, A., Menze, B., Diepold, K., & Butz, A. (2016). Humandrone-interaction: A case study to investigate the relation between autonomy and user experience. In Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8–10 and 15–16, 2016, Proceedings, Part II 14 (pp. 238–253). Springer International Publishing. Crandall, J. W., Goodrich, M., Olsen Jr., D. R., Nielsen, C. W., et al.: Validating
human-robot interaction schemes in multitasking environments. IEEE Trans. Syst. Man Cybern. 35(4), 438–449 (2005). Freedy, A., DeVisser, E., Weltman, G., & Coeyman, N. (2007). Measurement of trust in human-robot collaboration. In 2007 International symposium on collaborative technologies and systems (pp. 106–114). IEEE Goodrich, M. A., Schultz, A. C.: Human-robot interaction: a survey. Foundations
Trends Hum. Comput. Interact. 1(3), 203–275 (2007). Matejka, J. (2020). Robot as a member of combat unit a utopia or reality for ground forces?. Advances in Military Technology, 15(1), 7–24. Okkonen, J., Syvänen, A., Laarni, J., Viteli, J., Wahlström, J., Laine, M. (2024) Sense
of technology self-efficacy and motivation coexisting with algorithmic thinking. IHIET-AI 2024. AHFE Open Access DOI: 10.54941/ahfe1004566. Rossiter, A. (2020). Bots on the ground: an impending UGV revolution in military affairs?. In Robotics, Autonomous Systems and Contemporary International Security (pp. 161–183). Routledge. Rödel, C., Stadler, S., Meschtscherjakov, A., & Tscheligi, M. (2014, September).
Towards autonomous cars: The effect of autonomy levels on acceptance and user experience. In Proceedings of the 6th international conference on automotive user interfaces and interactive vehicular applications (pp. 1–8). Svendsen, G. B., Johnsen, J. A. K., Almås-Sørensen, L., & Vittersø, J. (2011).
Personality and technology acceptance: the influence of personality factors on the core constructs of the Technology Acceptance Model. Behaviour & Information Technology, 32(4), 323–334. https://doi.org/10.1080/0144929X.2011.553740 Wohleber, R. W., Stowers, K., Barnes, M., & Chen, J. Y. (2023). Agent transparency in mixed-initiative multi-UxV control: How should intelligent agent collaborators speak their minds?. Computers in Human Behavior, 148, 107866. Xu, W. (2020). From automation to autonomy and autonomous vehicles: Challenges
and opportunities for human-computer interaction. Interactions, 28(1), 48–53.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Design, Engineering, and Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-35-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional