Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Human values assessment toward AI-based patient state prediction

1 Data Science Design Team, ADSP, R-IH, RIKEN, Wako, 3510198, Japan ↗
2 Shirakawa Hospital, Omuta, 8370926, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1004600

Abstract

Recent advances in artificial intelligence (AI) technology are remarkable, and AI may predict a patient’s future state (prognosis) using large amounts of data in the future. However, patients are not always satisfied with the prediction results. In order for a patient to accept prediction results and change behaviors in life, prediction results should reflect his/her values. AI-based patient state prediction should be implemented in a way that is not only data-driven but also integrated with domain knowledge about human values. This paper reports a case study where we assessed data related to values toward building a domain knowledge for AI-based patient state prediction.We designed a rehabilitation exercise service based on a person-centered principle [1] and intervened with one patient [2]. From the preliminary stage of service design, we attempted to build rapport and motivate her. We expected her to self-disclose through repeated dialogues. While understanding her life backgrounds and values, we defined rehabilitation goals for her independent living.12 rehabilitation exercise sessions were held for three months, and questionnaires and semi-structured interviews were conducted before the first session and after the final session. Even after the experiment ended, we continued to have conversations with her and obtained information about her life after the experiment ended. Measurements taken after the experiment showed no noticeable effect on her physical functions. Regarding her self-disclosure, as the experiment progressed, she revealed more of her true feelings, including negative ones. Before the experiment, she negatively commented about daily activities that she could not do well, but after the experiment, she positively did with focusing on the things she could do, although she was not perfect. We confirmed her active participation in social activities, such as going out wearing a clothe made by sewing, a task that she had viewed negatively before the experiment. Factors that influenced her behavior changes include building rapport through repeated conversations and self-disclosure as an effect of the experimenter’s interest in and empathy for her. Regarding self-disclosure, at the beginning of the experiment, most disclosures were positive-sounding superficial ones, but gradually they changed to deeper disclosures that included negative content.The patient’s value data suggested by the self-disclosure in this study is still insufficient to form a value domain knowledge system. However, we believe that the acquisition of value data based on a patient’s self-disclosure and behavioral changes will help realize AI state prediction that will lead to patient acceptance in the future. Future work will include a detailed analysis of the factors that influenced self-disclosure and the construction of a value assessment framework.[1] Kitwood, T. and Bredin, K. (1992) Towards a theory of dementia care: Personhood and well-being, Ageing and Society, Vol.12, No.3, pp.269-287.[2] Yukihira, T et al. (2023). Toward an online rehabilitation exercise service based on personal independent living goals and risk management. Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications, Vol. 112, pp.187-194.

INTRODUCTION

Advances in AI technology may change the standard of healthcare services in the future. A patient’s future state is influenced by various factors, such as the medical prescription presented and the patient’s behavioral changes in daily life. Therefore, there are various patterns of possible future states of patients depending on the clinician’s judgement or the patient’s own choices. AI-based patient state prediction will change prescription and choice.

PatientsLikeMe (Wicks et al., 2010) is a service that allows patients to find data about other patients with similar symptoms, which provides patients with opportunities to make decisions and change their behavior. In the future, more advanced AI technology will be able to predict a patient’s state using large amounts of data obtained from other patients. In order for patients to effectively change their behavior, it is desirable that the prediction results from AI reflect the values of each patient. Although AI is a technology that

© 2024. Published by AHFE Open Access. All rights reserved. 482

Human Values Assessment Toward AI-Based Patient State Prediction 483

enables data-driven services, it is expected to enable services that take into consideration not only data utilization but also human values. This paper reports a case study of a values assessment conducted to provide insights into the development of future AI technologies.

ISSUES ADDRESSED Advances in technology bring convenience to human life, but they also have the potential to define the future in a way that narrows human possibilities. AI-based patient state prediction will present possible future state patterns based on highly accurate predictive calculations. The predicted results pre- sented may be pleasing to one patient but may not be acceptable to another. This is because each patient has a different sense of value and happiness. Therefore, it is expected that AI will suggest state transition destinations that take into account not only the possibility of curing diseases but also the level of happiness (See Figure 1). Each patient should be able to select State B, which is a state transition destination that will lead to a higher level of happiness, not just based on the criterion of living a long life.

Figure 1: Patient’s state transition for higher level of happiness.
Original PDF Figure 1:

There are several examples of challenges in healthcare research that are based on the use of AI in the future. Thieme introduced a human-centered approach to developing an AI application that predicts treatment outcomes for patients who are receiving human-supported, internet-delivered Cognitive Behavioral Therapy (iCBT) for symptoms of depression and anxiety (Thieme et al., 2022). Sivaraman focused on the clinician side and explored how clin- icians interact with real AI-based treatment recommendations in a setting where sequential treatments can affect outcomes in complex ways (Sivaraman et al., 2023). Pandl focused on values of data and analyzed the suitability of more scalable data valuation methods to value extensive amounts of medical imaging data (Pandl et al., 2021). Gyldenkaerne pointed out that there lies a need to balance the introduction of AI with a proper focus on the patients’ and the clinicians’ interests (Gyldenkaerne et al., 2020). Ayobi introduced a

co-design project that focused on personal health and included AI researchers as members (Ayobi et al., 2021). Regarding human values, Proynova ana- lyzed personal values on requirements for the development of health care information systems (Proynova et al., 2011). Although various types of AI- based healthcare research are being conducted, AI technology that can handle human values has not yet been realized. AI-based patient state prediction should be implemented in a way that is not only data-driven but also inte- grated with domain knowledge on human values. This paper introduces a case study where we assessed a patient’s self-disclosure data related to val- ues with the aim of building domain knowledge for AI-based patient state prediction.

DEVELOPING ONLINE REHABILITATION EXERCISE SERVICE As a case study, we designed an online rehabilitation exercise service based on the principle of person-centered care (Kitwood et al., 1992) and intervened with one patient (Yukihira et al., 2023). Being able to participate in rehabil- itation not only at a nursing facility but also in one’s own home will make more care services possible under the labor shortage situation in the nursing care domain. Person-centered care is a principle for deeply understanding each patient, and there is a practical framework for the care, called DCM, Dementia Care Mapping (University of Bradford). However, it is difficult to operate under the framework at care sites due to there being too many oper- ations in the DCM cycle. Thus, it cannot be expected that insights obtained from the DCM operations will be utilized for AI technology development. Person-centered care is derived from the context of dementia care but can be applied to other health care domains because it considers each patient’s sense of value. Each patient has their own unique life background and values; thus, care services should be designed in consideration of them so that the services can support each patient in independent living.

EXPERIMENT We designed a rehabilitation intervention that considered the patient’s inde- pendent living goals and factored in the person-centered principles, risk management, and service operations performed by care workers in the future. This design is based on an assessment framework that is data-driven and person-centered (Ihara et al., 2023). A preliminary experiment was conducted for the purpose of confirming the effect of the intervention and extracting issues for improving the service design. The subject in this experiment was a female 78 years old with left hemiplegia due to a stroke who receives outpa- tient rehabilitation once a week but desires more rehabilitation opportunities. The reasons for choosing her as a subject were her acceptance of informa- tion technology devices, her home Wi-Fi environment, and her willingness to cooperate based on an understanding of the purpose of the experiment. The experiment was conducted with informed consent, and a reward was paid after the experiment.

Human Values Assessment Toward AI-Based Patient State Prediction 485

Design of Rehabilitation Exercise In this experiment, an instructor performed model exercises in front of a large display at the nursing facility, and the subject could exercise by watching on a tablet screen at home that was connected to the facility (Figure 2). Each 45-minute exercise session consisted of a 20-minute first exercise, a 5-minute break, and a 20-minute second exercise. In addition to a brief medical inter- view, blood pressure and saturation of percutaneous oxygen were measured before and after each exercise session. Once a week, a total of 12 exercise sessions were performed for three months. Note that during this experiment, the subject continued to receive outpatient rehabilitation. In defining the con- tents of the exercise, first, based on preliminary information obtained from interviews with the subject and information sharing from the director of the facility, goals were set for independent living, such as being able to travel again and being able to cook again. After that, the body movement of the sub- ject was checked in person, and the content of the exercises was tentatively determined. An occupational therapist served as an instructor for the first to fourth session in order to adjust the load appropriately while observing the subject’s condition. From the 5th to the final session, on-site staff of the facility acted as an instructor in order to experience the service in operation.

Figure 2: Online rehabilitation exercise.
Original PDF Figure 2:

EVALUATIONS To evaluate the effects of the intervention, physical function measurements, questionnaires, and interviews were conducted.

Physical Function Measurements The physical function measurements in this experiment were based on the Cardiovascular Health Study frailty index, which considers weight loss, slow gait speed, low physical activity, exhaustion, and low grip strength. The mea- surements were taken monthly during the 3-month experiment, including at the beginning and end. As a result, no significant change was observed in the results for each index.

Questionnaires The subject was given a questionnaire at the beginning and end of the experiment. As for the questions, we referred to the ICF, the International Classification of Functioning, Disability and Health (World Health Organiza- tion, 2001), and we set eight items: recent body function, fatigue, motivation, number of recent outings, daily activities, role at home, hobbies, and social connection. Regarding the questionnaire at the end of the experiment, free- text items were added for answering whether participating in the online rehabilitation made daily life colorful or not and whether it was good to participate or not.

Interviews To get details on the questionnaire responses, the second author of this article conducted a semi-structured interview with the subject. We transcribed the recorded interview data and conducted a qualitative analysis using M-GTA, the Modified Grounded Theory Approach (Kinoshita, 2007). As a result, we obtained 19 concepts and 9 categories: current health status, recovered/un- recovered from paralysis, independent living, self-efficacy, motivation for rehabilitation, effects of physical functions, mental effects, and no effects (Yukihira et al., 2023). Even after the experiment ended, we continued to have conversations with the subject and obtained information about her life after the experiment. The content of the chats was recorded by the second author on the same day.

ANALYSES From the preliminary stage of service design, we attempted to build rapport and motivate the subject. We expected her to self-disclose through repeated dialogues. While understanding her life backgrounds and values, we defined rehabilitation goals for her independent living. We analyzed self-disclosure utterances included in the narratives told by her in the interviews and chats.

Self-Disclosure Before Experiment In interviews and chats before participating in the experiment, she often talked about her hobbies, such as cooking, driving, and gardening. It is said that self-disclosure related to hobbies is relatively easy to disclose to anyone, so it is understandable that hobbies were used as a topic in conversations. In the preliminary interview, there were many questions regarding her current situation, so her self-disclosure included many statements about inconvenient

Human Values Assessment Toward AI-Based Patient State Prediction 487

situations such as cooking and physical fatigue. She also expressed a desire to take action, such as by saying that she wanted to go on a trip to her favorite city, Kyoto. In general, it is difficult to disclose one’s physical self, such as inconveniences in daily activities, to an unfamiliar conversation partner, and she had partially disclosed this in the preliminary interview. Also, during a chat before the experiment began, we noticed that she placed her right hand over the back of her paralyzed left hand. This gesture may have been the result of her reluctance to disclose herself to us, the visitors, with whom she had not yet established rapport.

Self-Disclosure During Experiment After the experimental period began, at the request of the subject, the second author attended 7 of the 12 exercise sessions at her home as an observer. They chatted about both rehabilitation and personal topics during the 5-minute break of the exercise session and for about 30 minutes after the session. In the chat during those exercise session visits, in addition to disclosing her hobbies such as knitting, she made a number of mental disclosures. Specifically, she talked about her past experiences of her efforts to recover, as well as her thoughts on rehabilitation and participating in this exercise experiment. She also talked about her life with paralysis and disclosed her physical self.

Self-Disclosure After Experiment In the post-interview, questions were asked to compare her mental and physi- cal conditions before and after the experiment. She talked about her recovery over the past two years, as well as her current inconvenient situation, her thoughts on exercise, and her hopes for recovery. These are self-disclosures that included disclosure of both the mental self and the physical self. Even though she was aware that the speed of recovery would be slow, she said that she still wanted to go on a trip. Before the experiment, she commented nega- tively on daily activities that she could not do well, but after the experiment, she commented positively with a focus on the things she could do, although not perfectly.

Follow-Up Conversations After Experiment In a chat after the exercise experiment ended, she talked about the increase in the number of times she went out and spoke positively about her expec- tations toward recovery. She talked about an episode in which she started attending dressmaking school again; in other words, she was disclosing her existential self. She also disclosed her social self by introducing an episode in which she appeared on a TV news program and her kinship self by talking about her childhood through a memory of her parents. These findings suggest that building rapport led to various self-disclosures from her.

We confirmed her active participation in social activities, such as going out wearing clothes that she had sewed, a task that she had viewed nega- tively before the experiment. Factors that influenced her behavior changes

include building rapport through repeated conversations and self-disclosure as an effect of the experimenter’s interest in and empathy toward her. Regard- ing self-disclosure, at the beginning of the experiment, most disclosures were positive-sounding superficial ones, but they gradually changed to deeper disclosures that included negative content.

VALUE-BASED PATIENT STATE PREDICTION Values From Self-Disclosure

The various selves disclosed by the subject in this exercise experiment are related to her sense of values. Of course, her hobbies, which are relatively easy to express in conversation, reflect her values. She must think that trav- eling is a valuable activity, as it allows her to connect with society and independently engage in a variety of experiences that she would not be able to have at home. This self-disclosure about her hobbies suggests that she prioritizes activeness, autonomy, and extroversion as high values. The disclo- sure of strong expectations toward recovery through rehabilitation suggests a value in aspiring to live a life without physical inconvenience like before suffering from paralysis. In addition, her disclosure of past episodes of her efforts to recover and her own thoughts on rehabilitation suggest a value in highly appreciating the significance of achieving goals through one’s own efforts.

Utilizing Values in AI Technology Development The patient’s value data suggested by the self-disclosure in this study is still insufficient to form a value domain knowledge system. However, we believe that the acquisition of value data based on a patient’s self-disclosure and behavioral changes will help realize AI state prediction that will lead to patient acceptance in the future. The current status of our project is in the early stages of science for patient state prediction models. Instead of the AI predicting the state transition destination, this study is equivalent to the experimenter setting the state transition destination expected as a result of rehabilitation intervention as a rehabilitation goal for independent living through interviews etc. By evaluating the effectiveness of the intervention, we extracted data on the patient’s values from her self-disclosure and behavioral changes. Such extracted data is expected to be used as a model or dataset for state prediction that reflects individual values in the service ecosystem shown in Figure 3. One of the issues in developing AI technology is the inter- pretability of prediction models (Hong et al., 2020). We aim to realize an interpretable surrogate model that makes predictions under the constraints of the human mind. Since the performance of a model depends on the dataset used, we would like to improve the quality of predictions by utilizing a well- organized set of data on patients’ sense of values. Furthermore, in order for patients to accept the predictions of state transitions presented by the model and to change their behavior, we believe that the interface between the model and patients needs to be improved.

Human Values Assessment Toward AI-Based Patient State Prediction 489

Figure 3: AI-based service ecosystem based on evidence and narratives.
Original PDF Figure 3:

CONCLUSION

This paper introduced a case study of value assessment toward developing a technology for AI-based patient state prediction. We believe that the con- tribution of this study is that, through a patient intervention experiment, we induced a variety of self-disclosures, from superficial self-disclosure to deep self-disclosure, and assessed data regarding human values. Future work will include a detailed analysis of the factors that influence self-disclosure and the construction of a value assessment framework.

ACKNOWLEDGMENT

We thank the subject and care workers at the outpatient rehabilitation facility “Magokoro” in Omuta city, Japan who cooperated in the experiment.

REFERENCES

Ayobi, A., Stawarz, K., Katz, D., Marshall, P., Yamagata, T., Santos-Rodriguez, R., Flach, P. and O’Kane, A. A. (2021). Co-Designing Personal Health? Multidis- ciplinary Benefits and Challenges in Informing Diabetes Self-Care Technologies, Proc. of the ACM on Human-Computer Interaction, Vol. 5, No. 457, pp. 1–26. Gyldenkaerne, C., From, G., Mønsted, T. and Simonsen, J. (2020). PD and The

Challenge of AI in Health-Care, Proc. of the 16th Participatory Design Conference 2020, Vol. 2, ACM, pp. 26–29. Hong, S. R., Hullman, J. R. and Bertini, E. (2020). Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs, Proc. of the ACM on Human-Computer Interaction, Vol. 4, No. 68, pp. 1–26. Ihara, M., Tokunaga, H., Murakami, H., Saruwatari, S., Takeshita, K., Koga,

A., Yukihira, T., Hisano, S., Maeda, R. and Motoe, M. (2023). A data-driven but person-centered assessment framework for sustainable rehabilitation services, Proc. of Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems, Vol. 69, pp. 408–416. Kinoshita, Y. (2007). Modified grounded theory approach, https://m-gta.jp/en/. Kitwood, T. and Bredin, K. (1992). Towards a theory of dementia care: Personhood and well-being, Ageing and Society, Vol. 12, No. 3, pp. 269–287.

Pandl, K., Feiland, F., Thiebes, S. and Sunyaev, A. (2021). Trustworthy machine learning for health care: scalable data valuation with the shapley value, Proc. of the Conference on Health, Inference, and Learning (CHIL’21), ACM, pp. 47–57. Proynova, R., Paech, B., Koch, S. H., Wicht, A. and Wetter, T. (2011). Investigat-

ing the influence of personal values on requirements for health care information systems. Software Engineering in Health Care, Proc. of the 3rd Workshop on Software Engineering in Health Care (SEHC’11), ACM, pp. 48–55. Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M. and Perer, A. (2023). Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care, Proc. of the 2023 CHI Conference on Human Factors in Computing Systems (CHI’23), ACM, No. 754, pp. 1–18. Thieme, A. Hanratty, M., Lyons, M., Palacios, J., Marques, R., Morrison, C.

and Doherty, G. (2022). Designing Human-Centered AI for Mental Health: Developing Clinically Relevant Applications for Online CBT Treatment. ACM Transactions on Computer-Human Interaction, Vol. 30, Issue 2, No. 27, pp. 1–50. University of Bradford. Dementia Care Mapping. https://www.bradford.ac.uk/dem entia/training-consultancy/. Wicks, P., Massagli, M., Frost, J., Brownstein, C. A., Okun, S., Vaughan, T. E.,

Bradley, R. and Heywood, J. (2010). Sharing Health Data for Better Outcomes on PatientsLikeMe, Journal of Medical Internet Research, Vol. 12, No. 2: e19. World Health Organization (2001). International Classification of Functioning, Disability and Health, Geneva, https://www.who.int/classifications/icf/en/. Yukihira, T., Tokunaga, H., Ihara, M., Murakami, H., Koga, A., Maeda, R.,

Saruwatari, S., Takeshita, K., Hisano, S. and Motoe, M. (2023). Toward an online rehabilitation exercise service based on personal independent living goals and risk management, Proc. of Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications, Vol. 112, pp. 187–194.

Cover of the proceedings volume Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications

Explore the complete digital volume

Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-96-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →