INTRODUCTION
Robots are automating a wide range of professional activities in contem- porary industries and services, such as healthcare, manufacturing, military operations and service industries. This trend is also reflected by steady gro- wth in robot installations in industries around the world, with a prevalence in China, followed by Japan, USA, and then Europe (IFR, 2022).
Alongside the installation of new robots, industries are changing and chal- lenging how work is going to be performed. Critically, unlike traditional industrial robots in Industry 3.0 where robots operated in separate spaces on the shop floor with minimal physical interaction with humans, in the so-called Industry 4.0, industries are moving towards cyber-physical systems integration and human-robot collaboration (HRC). HRC requires humans
© 2023. Published by AHFE Open Access. All rights reserved. 31
and robots to interact closely and collaborate with shared tasks in shared (physical) spaces towards a common goal. Collaborative robots (or cobots) are robots designed to assist workers in performing shared tasks in the same workspace, ‘intelligently’ interacting through either gesture or speech reco- gnition (Cohen, Shoval, Faccio, & Minto, 2022). Ergonomic, as well as productivity and efficiency benefits are expected in future industries thanks to the synergic collaboration of humans robots. However, a number of ethi- cal concerns are raised in HRC with impacts at micro-, meso-/organisational, and macro-/societal levels.
Ethical research on HRC regards all issues of safeguarding humans from unintended and potentially unethical risks and hazards associated with cobots (Decker, 2008). Within the HRC domain, the term Roboethics has been coined to refer to the social and ethical aspects of the design, deve- lopment, and employment of collaborative and intelligent robots (Steinert, 2014; Veruggio & Operto, 2017). Four different typologies have been sugge- sted. The first typology regards robots as machines, although sophisticated. In the second typology, robots may have an intrinsic ethical dimension to do good acts; ethical considerations in this classification are particularly rele- vant in the healthcare and social sectors. Humanoid and pet-type robots fall into this category. In the third typology, robots are seen as moral agents, and therefore have the ability to make decisions on their own. The last typology – fortunately, played only in the science fiction blockbuster movies – sees robots as an evolution of a new species, having autonomy, conscience, and greater intellectual dimensions (Steinert, 2014; Veruggio & Operto, 2017).
Although we are well aware that there is still no universal agreement as to what is ethically or morally correct, there is growing interest in develo- ping – and even standardising – robot ethics (BS8611:2016). This paper aims to provide a review overview of available literature in the field to map the eth- ical aspects and concerns that are discussed in the HRC domain, taking into account the above Roboethics typologies. Methodologically, it will employ a narrative approach to identify which ethical concerns are discussed within each typology. The paper will conclude by outlining an agenda for future ethical research in HRC, specifically where current gaps emerge.
METHODOLOGY
We adopted a narrative approach for organising and analysing the literature, to highlight the state-of-the-art on ethical research in HRC. Narrative revi- ews are suited for understanding the status of knowledge of the researched phenomenon, identifying gaps and inconsistencies, and highlighting the signi- ficance of new research (Fan, Breslin, Callahan, & Iszatt-White, 2022; Paré, Trudel, Jaana, & Kitsiou, 2015).
The review was carried out considering studies published in peer- reviewed journals, books, and grey literature using the following rele- vant keywords: ethics OR roboethics OR “moral agency” AND “human- robot collaboration” OR “human-robot interaction” OR cobot* OR “collaborative robot*” OR “social robot*” OR “assistive robot* OR “mili- tary robot*”.
Where Are We At? A Review of the State-of-the-Art of the Ethical Considerations 33
The databases searched were: Web of Science, Scopus, Science Direct, Ebsco, American Psychological Association databases (PsycARTICLES), Medline, Journal Storage (JSTOR), and Google Scholar. The paper results were screened from the abstract, and the selected papers were saved in the reference management software EndNotex9 (©Clarivate Analytics) and then uploaded in NVivo (v.12 Pro for Windows, ©QSR International) for analysis. The Qualitative Content Analysis method (Schreier, 2012) was employed to code the literature material into a concept-driven coding frame mirroring the suggested Roboethics typologies. The analysis was performed collaboratively by this paper’s co-authors.
Typology 1 – Robots Are Mere (Intelligent) Machines This typology refers to considering the robots as mere machines, although sophisticated. This approach takes an instrumental view of the human-robot interaction (HRI), and therefore robots are conceived with “amoral robot agency” (Asaro, 2006, p. 11). Roboethics within this frame can be compared to Engineering Applied Ethics (Veruggio & Operto, 2017). In this section, we will address HRC in the manufacturing and military sectors.
Within the manufacturing industry, specific ethical questions that blue- collar workers are facing with regard to shop floor HRC are still under- explored (Fletcher & Webb, 2017; van Wynsberghe & Roeser, 2022). Most of the ethical debates around the industries of the future (4.0 and 5.0) centre around the concern of robots creating a jobless society, with labour displa- cement and/or replacement. Critically, the installation of robots will mainly aim to release the human workforce from those repetitive, heavy, and dan- gerous tasks, replacing manual tasks with more sophisticated cognitive and supervisory roles that will improve the efficiency and quality of industrial production (Moniz & Krings, 2016; Murphy, 2022). This trend will accele- rate the need for novel HRC business models, requiring more qualified and skilled workers to accomplish complex and collaborative task configurati- ons. Therefore, human operators will need to improve their technical and non-technical skills to keep up with the work demands (Rangraz & Pareto, 2020). Indeed, the literature is advocating for new roles involving workers in more creativity and problem-solving capabilities (Holford, 2019). Ethical considerations are posed on whether we shall assume that the employees of the future industrial robotics factories must all engage in more creative acti- vities to retain their jobs, instead of being comfortable performing routine and non-creative tasks (van Wynsberghe & Roeser, 2022). Additionally, in the future, flexible working schedules might require specific training to sup- port employees in managing their workloads and improving work-life and well-being (Martinetti, Chemweno, Nizamis, & Fosch-Villaronga, 2021).
Another well-known ethical concern in the HRC domain in industrial settings regards workers’ physical well-being and safety. ISO standards that cover HRC (ISO10218:2011 parts 1 and 2; ISO/TS 15066:2016) and industrial applications of Human-Machine Interaction (HMI) research are challenged to advance theoretical and methodological models able to inte- grate and predict the complex safety-related relationship and collaboration
of humans and robots at the factory floor. This includes not only the physical safety outcome of the actual human-robot dynamics, but also the psycholo- gical well-being of humans during the task-sharing interaction with robots (Fletcher & Webb, 2017). This regards the feelings and experiences human operators encounter in their relationship with robots, for which trust has been found to be an underlying key factor (Charalambous et al., 2016). Indeed, cobots may pose psychological harm (Fletcher & Webb, 2017), such as bur- nout and/or anxiety when interacting with them (van Wynsberghe & Roeser, 2022; Yam et al., 2022). In line with this, recent research is attempting to comprehend the role of emotions and embodied experiences associated with a subjective/perception of HRC (van Wynsberghe & Roeser, 2022).
Finally, the ethical issues of data sharing (regarding confidentiality and privacy) and performance data monitoring are pivotal in HRC in manufa- cturing settings (Fletcher & Webb, 2017; van Wynsberghe & Roeser, 2022). Industry 4.0 implies an ongoing exchange of data captured from sensors and robotics in interaction with humans on the factory floor and transferred to the middle and top management. How these data (involving human and robot performances at individual and collaborative levels) are managed still needs to be fully understood. Indeed, the ethical implications are to be fully represented to ensure that the captured human data not only comply with data protection protocols regarding informed consent, but also for which performance monitoring purposes these data are collected. The ethical impli- cations of performance data monitoring warrant additional research and policy guidelines within manufacturing/industrial settings to help clarify the extent of the trade-off between performance, quality and well-being resulting from HRC.
The academic literature on the use of robots in the military sector predomi- nantly addresses ethical questions focused on Lethal Autonomous Weapons Systems (LAWS), which are autonomous weapon systems that use robotics and artificial intelligence to identify, select, and kill targets without human intervention. These weapons have already been deployed on the battlefield (Rossiter & Cannon, 2022; Hwang & Song, 2022).
In contrast, other military robotic applications require collaboration with humans. Examples regard remote-controlled ground robots, in which robots are deployed for dull, dirty, and dangerous tasks such as saving lives by moving into potential danger zones ahead of or instead of humans and are changing modern warfare (Lin, Abney, Bekey, 2014). They perform tasks such as neutralising explosive devices, search and rescue in disaster areas and battlefields, equipment supply, surveillance and reconnaissance (through automated facial or image-recognition and object-detection capabilities), che- mical, biological, radiological, and nuclear (CBRN) detection, or sniper detection. Indeed, disarming and disposing of bombs is one of the most poten- tially hazardous professional activities, and therefore was one of the first practical applications for robotics. Since their implementation in the 1970s, bomb disposal robots have undergone a considerable transformation, espe- cially in their user control methods. Critically, modern Explosive Ordnance Disposal (EOD) robots are progressing towards virtual reality and advanced haptic sensory feedback controlled remotely to keep the operator safe.
Where Are We At? A Review of the State-of-the-Art of the Ethical Considerations 35
These military robots offer great opportunities for different mission sets to improve soldier safety and reduce their cognitive burden. However, this might pose a danger: “the more we rely on machines to relieve ourselves of cognitive responsibilities, the more we forget how to do important things” (Jordan, 2016, p. 225). As an extent of military troops, the robots must not impose any extra workload on the team (Sanaullah, Akhtaruzzaman & Hossain, 2022).
Moral and ethical considerations in the HRC domain in military settings concern the worry that military operators can put too much trust in the robots. Those systems might not be able to adapt to the inevitable com- plexities of war. Critically, it is hard to differentiate an anomaly from a threat, distinguish valid from invalid targets and decide who is accounta- ble for mistakes. Further, sensor data might be imperfect, with errors of both systematic and random nature (Ha, Yen & Balaguer, 2019). Robot safety and security are also vital concerns. Because a robotic system has sensors, actua- tors, drivers and controllers, it is vulnerable to hardware failure or intrusions such as virus attacks, hacking, or robot worms that block sensitive parts of the sensors and produce fake signals (Sanaullah, Akhtaruzzaman & Hossain, 2022). Finally, with fewer casualties among the own servicemen and service- women, robots could lower the cost of warfare, making the conflict between nations more likely.
Overall, the opinion in the literature is that no matter what kind of military robots we deal with (manual, semi-autonomous, autonomous), they must be supervised and human-controlled.
Typology 2 – Robots Have Ethical Dimensions Ethical considerations of HRC in this classification are particularly relevant in the healthcare and social sectors, involving target patients, healthcare operators, and caregivers.
Socially assistive and companion robots are conceived and designed to engage on an emotional level because they are able to trigger social and emo- tional responses from their human users (de Graaf, 2016; Steinert, 2014). Indeed, they are often referred to as “affective humanoid social robots” (Shaw-Garlock, 2009; Steinert, 2014). Humanoid and zoomorphic robots (or robopets) may fall into this second typology (Steinert, 2014). Humanoid robots with anthropomorphic features can mimic various social interactions with humans, including natural communication, social learning and coope- ration (Breazeal, 2004). Critically, they are designed based on the rules of human–human interaction to behave as if they have mental states, persona- lities and intentions (de Graaf, 2016; Steinert, 2014), and as such, be able to simulate feelings and act accordingly (Veruggio & Operto, 2017). For this reason, they may be considered “ethically considerable beings” (Steinert, 2014, p. 252). These robots have started to be introduced in recent years to (1) aid patients in medical treatments, which could include interventions both in residential and in-home settings targeting healthcare patients, and/or adults with cognitive impairments such as dementia (Hebesberger et al., 2016; Pedersen, Reid, & Aspevig, 2018). Furthermore, (2) robots are developed
to help minimise older adults’ social isolation and as such, act as home com- panions (Ienca, Jotterand, Vic˘a, & Elger, 2016; Pedersen, Reid, & Aspevig, 2018). In relation to (1), socially assistive robots as an aid in treatment may be further classified in: rehabilitation robots (used mainly in the physical rehabilitation of patients) (Mohebbi, 2020); service robots (used to comple- ment the human caregivers in several direct cares, e.g. support of patients affected by dementia in memory-related activities of daily life) (Ienca, Jot- terand, Vic˘a, & Elger, 2016); and telepresence robots (used to provide remote monitoring through telephony and long-range remote control). The primary human-robot interaction (HRI) functions of these robots encompass all service-related requirements, such as dealing with emergencies, nutritional support, physical and memory-related assistance through cognitive or emo- tional stimulation, logging daily activities, and collaboration with healthcare staff, including exchanging data (Kim et al., 2021).
Regarding (2), social companion robots have been implemented to reme- diate social isolation among older adults and, therefore, enhance communi- cational and emotional capabilities with their users. Additionally, robopets are able to elicit and develop affective social interactions (see, for example, studies around companion-dog robots: (Konok, Korcsok, Miklósi, & Gácsi, 2018;) Krueger, Mitchell, Deshpande & Katz, 2021; Miklósi & Gácsi, 2012). Thus, human-animal interactions have been used as inspirations to recreate and build behaviour models for robopets (Miklósi & Gácsi, 2012). Indeed, animal behaviour is regarded to be simpler than human behaviour, and as such easier to design and implement in companion social robots.
The benefits of employing these assistive and companion robots are repor- ted in the literature, both for the caregivers themselves (Persson, Redmalm, & Iversen, 2022) and for patients, especially targeting vulnerable people (such as patients affected by dementia). Agitation reduction, loneliness decrease, and improvement in social interaction are the main results that have been published to date (e.g., Bemelmans, Gelderblom, Jonker, & de Witte, 2012; Van Orden et al., 2021). Additionally, they could support older adults in increasing their sense of control and autonomy (Sharkey & Sharkey, 2012).
Ethical considerations with regard to privacy and surveillance issues are extensively discussed in the literature. Here, the focus mainly centres on the possibility of assistive robots monitoring, following and storing the pati- ents’ daily activities around the home. Additional concerns are raised in cases where the patient’s mental state deteriorates further, and they become confused and/or panic in the presence of the robot (Sharkey & Sharkey, 2012). Another ethical concern regards the problem of ‘objectification’ of older adults, especially when involving assistive robots carrying out care tasks (Vandemeulebroucke et al., 2020). Indeed, greater attention should be given to prioritising older patients’ welfare and health, besides the additional interest of reducing the caregivers’ workload (and costs) (Decker, 2008; Sharkey & Sharkey, 2012).
Ethical concerns have been raised in relation to the feelings and emotions of attachment/detachment that users may experience in their interaction with
Where Are We At? A Review of the State-of-the-Art of the Ethical Considerations 37
companion robots. This issue was specifically investigated in patients affe- cted by dementia, where the ethical challenge of deception emerged. Some patients misperceived the robopets as real animals and displayed feelings of attachment towards them (Koh, Ang, & Casey, 2021; Sharkey & Sharkey, 2012). The potential for deception should be minimised wherever possible, and caregivers should clearly present the robopet as a robot and refrain from referring to it as a real animal (Koh, Ang, & Casey, 2021).
Typology 3 – Robots Possess Moral Agency This typology refers to those semi or fully autonomous robots that are desi- gned, and have the ability to make decisions on their own, with no human intervention to take over (Floridi & Sanders, 2004; Steinert, 2014; Veruggio & Operto, 2017). Examples here may include the autopilot in aircraft, Lethal Autonomous Weapon Systems (LAWS), and self-driving cars.
How these robots are ‘equipped’ with a moral agency is not only the responsibility of the researchers/engineers and programmers, who might not master the appropriate ethical knowledge (Gordon, 2020). Many factors are at stake, which need to be modelled (the context, the environment, other relevant conditions, unforeseen circumstances, etc.). Currently, trolley dilem- mas have been used to collect human decisions worldwide on several moral preferences and dilemmas, and how cultural differences may influence these decisions (Gold, Colman, & Pulford, 2014). Although there is still disagre- ement in the literature about whether trolley dilemmas represent the actual ethical challenge (Anderson & Anderson, 2011), these are employed to illu- strate the morality concerns the future autonomous robot/automation shall have.
Critically, the design of moral agents should incorporate the dimension of ‘autonomy’ (i.e. the possibility of free action), the ‘sensibility to values and norms’ to act ‘intentionally’ for the greater good, and the ‘responsibility’ (Steinert, 2014; Sullins, 2006). The challenges raised concerning the attribu- tion of ‘responsibility as accountability’ (Smith, 2015; Watson, 1995) are a major concern at professional (e.g., engineers, human factors experts), manu- facturer, robot (e.g., if they are built with a ‘conscience’), legal and societal levels. Indeed, the ethical liability of artificial moral agents regards answ- ering questions related to who is responsible and for what and for whom. Smith (2015) has argued that responsibility is about answerability: “a morally responsible agent is one who can intelligibly be asked to ‘answer for’ her attitudes and conduct” (p. 104).
Typology 4 – Robots Are a New Species Many early science-fiction films portrayed robots as an evolution of a new species, having autonomy, conscience, and greater intellectual dimen- sions. These fictional robots would take over the world, turning humans into enslaved people (Coeckelbergh, 2022; Gunkel, 2018). Nowadays, we refer to this type of robotics as depending on Artificial Super Intellige- nce (ASI), a type of AI that surpasses human intelligence and can perform
any task better than a human. It self-learns and evolves with a consci- ousness of its own. ASI outperforms humans to achieve societal objectives and facilitate space exploration but can also threaten the very existence of the human race (Kanade, 2022). Luckily, these dystopian predictions have not materialised. Instead, we have seen robots being designed to become companions and collaborators. However, consumers may soon find it more challenging to distinguish embodied robots from human employees (Schmitt, 2020).
Recent literature shows that robotic science and engineering evolve rapidly with the expansion of biohybrid robotics research. Robots are being powered by living muscle tissue or cells. These biobots are safer around people and less harmful to their work environment (Webster-Wood et al., 2022; Mazzolai & Laschi, 2020). There is also continued progress in cyborg-type robots (Yokota et al., 2023; Heffernan, 2019).
Philosophers apply a moral approach to robot development, wondering how we should respond to robots that look like humans and behave as if they are alive. Prescott (2018) points out that “robots are a new kind of entity, not quite alive and yet something more than machines”. Although still in its infancy, these developments will lead to new ethical dilemmas and questions. Is it acceptable to kick a robot? Should robots have rights? (Gunkel, 2018; Gordon & Pasvenskiene, 2021, Lima et al., 2020).
FUTURE RESEARCH AGENDA AND CONCLUSION Since Asimov’s famous novel on the laws of robotics, moral and ethi- cal considerations have been greatly discussed in the literature; however, applications have been sparse. Critically, recent studies have started to inve- stigate the extent to which ethics or moral action are to be designed into systems, whatever the typology considered. Much discussion follows regar- ding responsibility if something goes wrong (however that is judged), which is shared across autonomous systems and, much like human error, often judged with the benefit of hindsight.
We may reach a point where decisions are made quickly, excluding humans from the process. This will require a new level of trust in those systems. Moving the human to a supervisory role brings familiar challenges in terms of maintaining situational awareness, for example.
We know there is a skills shortage, particularly in tech companies. Work in this area can inform future job roles, work design and skills needed. We know that the future roles will be different to the ones we have now, and we need to start understanding a more appropriate approach to system design rather than simply replacing human roles with robots. They are different, and a successful system will exploit and enhance those differences.
Considering robot ethics should help us reflect on our approach in design, taking a user-centred approach and moving away from ‘can we’ to ‘should we’.
Where Are We At? A Review of the State-of-the-Art of the Ethical Considerations 39
ACKNOWLEDGMENT
The present work is supported by EPSRC (Engineering and Physical Scie- nces Research Council) and ISCF (Industry Strategy Challenge Fund) under the Made Smarter scheme No EP/V062158/1 Made Smarter Innovation - Research Centre for Smart Collaborative Industrial Robotics.
REFERENCES
Anderson, S. L., & Anderson, M. (2011). Machine Ethics. United States: Cambridge University Press. Asaro, P. (2006). What Should We Want From a Robot Ethic? International Review
of Information Ethics, 6(12), 9–16. doi: https://doi.org/10.29173/irie134. Bemelmans, R., Gelderblom, G. J., Jonker, P., & de Witte, L. (2012). Socially Assistive Robots in Elderly Care: A Systematic Review into Effects and Effective- ness. Journal of the American Medical Directors Association., 13(2), 114–120.e1. https://doi.org/10.1016/j.jamda.2010.10.002 Breazeal, C. (2004). Designing Sociable Robots. Boston, MA: MIT Press. BS861:2016. Robots and robotic devices. Guide to the ethical design and application
of robots and robotic systems. Charalambous, G., Fletcher, S., & Webb, P. (2016). The development of a scale to evaluate trust in industrial human-robot collaboration. International Journal of Social Robotics, 8, 193–209. Coeckelbergh, M. (2022). Robot ethics. MIT Press. Cohen, Y., Shoval, S., Faccio, M., & Minto, R. (2022). Deploying cobots in collaborative systems: major considerations and productivity analy- sis. International Journal of Production Research, 60(6), 1815–1831. doi: 10.1080/00207543.2020.1870758 Decker, M. (2008). Caregiving robots and ethical reflection: the perspective of inter-
disciplinary technology assessment. AI & Soc 22, 315–330. https://doi.org/10. 1007/s00146-007-0151-0 de Graaf, M. M. A. (2016). An Ethical Evaluation of Human–Robot Relationships.
International Journal of Social Robotics, 8(4), 589–598. doi: 10.1007/s12369- 016-0368-5. Fan, D., Breslin, D., Callahan, J. L., & Iszatt-White, M. (2022). Advancing literature review methodology through rigour, generativity, scope and transparency. Inter- national Journal of Management Reviews, 24(2), 171–180. doi: https://doi.org/ 10.1111/ijmr.12291. Fletcher, S. R., & Webb, P. (2017). Industrial Robot Ethics: The Challenges of Closer
Human Collaboration in Future Manufacturing Systems. In M. I. Aldinhas Fer- reira, J. Silva Sequeira, M. O. Tokhi, E. E. Kadar, & G. S. Virk (Eds.), A World with Robots: International Conference on Robot Ethics: ICRE 2015 (pp. 159–169). Cham: Springer International Publishing. Floridi, L., Sanders, J. (2004). On the Morality of Artificial Agents. Minds and Machines 14, 349–379. https://doi.org/10.1023/B:MIND.0000035461.63578.9d Gold, N., Colman, A., & Pulford, B. (2014). Cultural differences in responses to
real-life and hypothetical trolley problems. Judgment and Decision Making, 9(1), 65–76. doi: 10.1017/S193029750000499X. Gordon, J. S. (2020). Building Moral Robots: Ethical Pitfalls and Challenges. Sci Eng Ethics 26, 141–157. https://doi.org/10.1007/s11948-019-00084-5 Gordon, J. S., & Pasvenskiene, A. (2021). Human rights for robots? A literature
review. AI and Ethics, 1(4), 579–591.
Gunkel, D. J. (2018). Robot rights. MIT Press. Ha, Q. P., Yen, L., & Balaguer, C. (2019). Robotic autonomous systems for earthmoving in military applications. Automation in Construction, 107, 102934. Kanade, V. (2022) Narrow AI vs. General AI vs. Super AI: Key Compa- risons. https://www.spiceworks.com/tech/artificial-intelligence/articles/narrow-ge neral-super-ai-difference/ Koertner, T., Gisinger, C., Pripfl, J., & Dondrup, C. (2016, March 07–10). Lessons
learned from the deployment of a long-term autonomous robot as companion in physical therapy for older adults with Dementia: A mixed methods study. Paper presented at the 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand. doi: 10.1109/HRI.2016.7451730. Heffernan, T. (2019). Fiction meets science: Ex Machina, artificial intelligence, and the robotics industry. Cyborg Futures: Cross-disciplinary Perspectives on Artificial Intelligence and Robotics, 127–140. Holford, W. D. (2019). The future of human creative knowledge work within the
digital economy. Futures, 105, 143–154. doi: https://doi.org/10.1016/j.futures. 2018.10.002. Hwang, W. J., & Song, S. H. (2022). The extension of Turkish influence and the use of drones. Comparative Strategy, 41(5), 439–458. Ienca, M., Jotterand, F., Vic˘a, C., & Elger, B. (2016). Social and Assistive Robotics in
Dementia Care: Ethical Recommendations for Research and Practice. Internatio- nal Journal of Social Robotics, 8(4), 565–573. doi: 10.1007/s12369-016-0366-7. IFR (2022). World Robotics 2022: Statistics, Market Analysis, Forecasts and Case Studies. Retrieved from: https://ifr.org/img/worldrobotics/Executive_Summary_ WR_Industrial_Robots_2022.pdf ISO 10218-1:2011. Robots and robotic devices — Safety requirements for industrial
robots — Part 1: Robots. ISO 10218-1:2011. Robots and robotic devices — Safety requirements for industrial robots — Part 2: Robot systems and integration. ISO/TS 15066:2016. Robots and robotic devices — Collaborative robots Jordan, J. M. (2016). Robots. MIT Press. Kim, J., Kim, S., Kim, S., Lee, E., Heo, Y., Hwang, C. Y., Choi, Y. Y., Kong, H. J., Ryu,
H., & Lee, H. (2021). Companion robots for older adults: Rodgers’ evolutionary concept analysis approach. Intelligent service robotics, 14(5), 729–739. https://do i.org/10.1007/s11370-021-00394-3 Koh, W. Q., Ang, F. X. H., Casey, D. (2021). Impacts of Low-cost Robotic Pets for Older Adults and People With Dementia: Scoping Review. JMIR Rehabilitation and Assistive Technologies, 8(1), e25340. doi: 10.2196/25340. Konok, V., Korcsok, B., Miklósi, Á., & Gácsi, M. (2018). Should we love robots? –
The most liked qualities of companion dogs and how they can be implemented in social robots. Computers in Human Behavior, 80, 132–142. https://doi.org/10. 1016/j.chb.2017.11.002 Krueger, F., Mitchell, K. C., Deshpande, G. et al. (2021). Human–dog relationships as a working framework for exploring human–robot attachment: a multidisciplinary review. Anim Cogn 24, 371–385. https://doi.org/10.1007/s10071-021-01472-w Lima, G., Kim, C., Ryu, S., Jeon, C., & Cha, M. (2020). Collecting the public
perception of AI and robot rights. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2), 1–24. Lin, P., Abney, K., & Bekey, G. A. (Eds.). (2014). Robot ethics: the ethical and social implications of robotics. MIT Press.
Where Are We At? A Review of the State-of-the-Art of the Ethical Considerations 41
Martinetti A., Chemweno P. K., Nizamis K., & Fosch-Villaronga E. (2021).
Redefining Safety in Light of Human Robot Interaction: A Critical Review of Current Standards and Regulations. Front. Chem. Eng. 3:666237. doi: 10.3389/fceng.2021.666237. Mazzolai, B., & Laschi, C. (2020). A vision for future bioinspired and biohybrid robots. Science robotics, 5(38), eaba6893. Miklósi, A. & Gácsi, M. (2012). On the utilisation of social animals as a model for
social robotics. Front. Psychology 3:75. doi: 10.3389/fpsyg.2012.00075. Mohebbi, A. (2020). Human-Robot Interaction in Rehabilitation and Assistance: a Review. Current Robotics Reports, 1(3), 131–144. doi: 10.1007/s43154-020- 00015-4. Moniz, A., & Krings, B.-J. (2016). Robots Working with Humans or Humans
Working with Robots? Searching for Social Dimensions in New Human-Robot Interaction in Industry. Societies, 6(3), 23. MDPI AG. Retrieved from https: //dx.doi.org/10.3390/soc6030023. Murphy, R. R. (2022). Co-bots get the good jobs while workers get a humanrobot interaction nightmare. Science Robotics, 7(65). doi: 10.1126/scirobo- tics.abp8431 Nejat G, Sun Y, Nies M. (2009). Assistive Robots in Health Care Settings. Home Health Care Management & Practice, 21(3), 177–187. doi: 10.1177/1084822308325695. Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015). Synthesising infor-
mation systems knowledge: A typology of literature reviews. Information & Management, 52(2), 183–199. doi: https://doi.org/10.1016/j.im.2014.08.008. Pedersen, I., Reid, S., & Aspevig, K. (2018). Developing social robots for aging populations: A literature review of recent academic sources. Sociology Compass, 12(6), e12585. doi: https://doi.org/10.1111/soc4.12585. Persson, M., Redmalm, D., & Iversen, C. (2022). Caregivers’ use of robots and their
effect on work environment – a scoping review. Journal of Technology in Human Services, 40(3), 251–277. doi: 10.1080/15228835.2021.2000554. Rangraz M. & Pareto, L. (2021). Workplace work-integrated learning: supporting industry 4.0 transformation for small manufacturing plants by reskil- ling staff, International Journal of Lifelong Education, 40:1, 5–22, DOI: 10.1080/02601370.2020.1867249 Rossiter, A., & Cannon, B. J. (2022). Turkey’s rise as a drone power: trial by fire.
Defense & Security Analysis, 38(2), 210–229. Sanaullah, M., Akhtaruzzaman, M., & Hossain, M. A. (2022). Land-robot technologies: The integration of cognitive systems in military and defence. NDC E-JOURNAL, 2(1), 123–156. Schreier, M. (2012). Qualitative Content Analysis in Practice. London: SAGE
Publications. Sharkey, A., & Sharkey, N. (2012). Granny and the robots: ethical issues in robot care for the elderly. Ethics and Information Technology, 14(1), 27–40. doi: 10.1007/s10676-010-9234-6. Shaw-Garlock, G. (2009). Looking Forward to Sociable Robots. International
Journal of Social Robotics, 1(3), 249–260. doi:10.1007/s12369-009-0021-7. Smith, A. M. (2015). Responsibility as Answerability. Inquiry, 58(2), 99–126. doi: 10.1080/0020174X.2015.986851. Schmitt, B. (2020). Speciesism: an obstacle to AI and robot adoption. Marketing
Letters, 31(1), 3–6. Steinert, S. (2014). The Five Robots—A Taxonomy for Roboethics. International Journal of Social Robotics, 6(2), 249–260. doi: 10.1007/s12369-013-0221-z.
Sullins, J. P. (2006). When is a robot a moral agent? International Review of Information Ethics 6 (12):23–30. Watson, G. (1996). Two Faces of Responsibility. In Agency and Answera- bility, ed. Gary Watson, pp. 260–286. Oxford: Oxford University Press, 2004.10.1093/acprof: oso/9780199272273.001.0001 Webster-Wood, V. A., Guix, M., Xu, N. W., Behkam, B., Sato, H., Sarkar, D.,... & Par-
ker, K. K. (2022). Biohybrid robots: recent progress, challenges, and perspectives. Bioinspiration & Biomimetics. Vandemeulebroucke, T, Dierckx de Casterlé, B., Welbergen, L, Massart, M., Gastmans, C. (2020). The Ethics of Socially Assistive Robots in Aged Care. A Focus Group Study With Older Adults in Flanders, Belgium, The Journals of Gerontology: Series B, 75(9), 1996–2007. https://doi.org/10.1093/geronb/gbz070 Van Orden, K. A., Bower, E., Beckler, T., Rowe, J. et Gillespie, S. (2022). The
Use of Robotic Pets with Older Adults during the COVID-19 Pandemic. Clinical gerontologist, 45(1), 189–194. doi: 10.1080/07317115.2021.1954122. van Wynsberghe, A., Ley, M., & Roeser, S. (2022). Ethical Aspects of Human– Robot Collaboration in Industrial Work Settings. In M. I. Aldinhas Ferreira & S. R. Fletcher (Eds.), The 21st Century Industrial Robot: When Tools Become Collaborators (pp. 255–266). Cham: Springer International Publishing. Veruggio, G., & Operto, F. (2017). Roboethics: a Bottom-up Interdisciplinary
Discourse in the Field of Applied Ethics in Robotics. In W. Wallach & P. Asaro (Eds.), Machine Ethics and Robot Ethics. London: Routledge. Yam, K. C., Tang, P. M., Jackson, J. C., Su, R., & Gray, K. (2022). The rise of robots increases job insecurity and maladaptive workplace behaviors: Mul- timethod evidence. Journal of Applied Psychology. Advanced online publication. https://doi.org/10.1037/apl0001045 Yokota, C., Tanaka, K., Omae, K., Kamada, M., Nishikawa, H., Koga, M.,... &
Minami, M. (2023). Effect of cyborg-type robot Hybrid Assistive Limb on patients with severe walking disability in acute stroke: A randomised controlled study. Journal of Stroke and Cerebrovascular Diseases, 32(4), 107020.