Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Human-Centered Design for a Virtual Human led mHealth Intervention for Suicide Prevention

USC Institute for Creative Technologies, Los Angeles, CA 90094, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1004118

Abstract

Addressing the significant mental and physical healthcare needs of Veterans requires innovative strategies to enhance access to evidence-based care. The integration of Virtual Human (VH) agents into Mobile Health (mHealth) applications presents a promising opportunity to overcome barriers associated with suicide prevention and connect with Veterans. The Battle Buddy (BB) project was conceived as a mobile wellness and suicide prevention application, empowering Veterans with an always-available resource concierged by an engaging and supportive conversational VH agent. Human-centered design is essential in the development of all interactions focused on the persuasive strategies of (1) personalization, (2) self-monitoring, (3) tunneling, (4) suggestion, and (5) expertise. Veterans can interact with the BB VH during daily check-ins, learn about mental health and wellness strategies, participate in interactive activities, increase self-awareness of their current status, and build and work safety plans in times of suicidal crisis. BB is designed to provide the Veteran with easy access to a suicide prevention ecosystem in which a wealth of evidenced-based interventions will be delivered in a non-stigmatizing fashion by a computer-based dialogue system with virtual embodiment, utilizing various multi-modal language cues such as text, speech, animated facial expressions, and gestures to interact with users. This paper explores our human-centered design process for the BB feature set to target the negative effects of social isolation and loneliness, conditions that challenge Veteran healthcare and suicide prevention.

INTRODUCTION

Suicide is a preventable public health problem. The latest National Suicide Prevention Veterans report identified over 6,000 Veteran deaths by sui- cide in 2020. This is an average of almost 17 deaths per day. Sadly, all odds point towards this being an underestimate. The Department of Veterans Affairs (VA) has put significant effort into reducing Veteran sui- cide, but 56% of Veterans who die by suicide have no contact with the VA (Prevention, 2022).

Emerging mobile health (mHealth) technologies, specifically mHealth applications (apps) designed to support mental health, are considered promis- ing tools for overcoming stigma and engaging service members in their own care (Tam-Seto et al., 2018). In fact, the National Center for Telehealth and

© 2023. Published by AHFE Open Access. All rights reserved. 98

Human-Centered Design for a Virtual Human Led mHealth Intervention 99

Technology and the Department of Veterans Affairs are releasing mHealth apps to deliver evidence-based mental health treatments. This rapid rise in interest is in part due to the ability of these apps to transform mobile and wearable devices into monitoring and therapeutic platforms that can capture mental health symptoms in real time and deliver on-the-go mental health support (Torous et al., 2017). Mobile apps also offer a viable option for mil- itary personnel to access care confidentially, anytime, anywhere -- reducing stigma-related barriers as well as occupational barriers to care, positioning these applications to be the future of suicide prevention (Tam-Seto et al., 2018). Despite the widespread consensus in the mHealth research community that identifying the needs and perspectives of targeted users is a vital component in good intervention development, human-centered design has not been widely applied in the domain of mHealth apps (Yardley et al., 2015). In fact, many current mHealth interventions are designed on the basis of existing health- care system constructs that may not be as effective when delivered in a novel medium without including end-users in the design process (Schnall et al., 2016). Development that neglects incorporating the voice of the user can adversely affect adherence, efficacy, and even full validation of interventions due to user attrition (Trifan et al., 2019). This is particularly concerning as interest in mHealth apps grows in both the military and the civilian sector, with current tallies of mental health offerings exceeding 10,000 apps (Torous et al., 2017).

Battle Buddy (Fig. 1) is a specialized AI-drivsen mobile health (mHealth) application tailored exclusively for suicide prevention in Veterans devel- oped at the University of Southern California’s Institute for Creative Tech- nologies sponsored by the US Army DEVCOM Soldier Center and the Department of Veterans Affairs (VA), in collaboration with SoldierStrong. The VH component of Battle Buddy is a computer-based dialogue sys- tem with virtual embodiment, utilizing various multi-modal language cues such as text, speech, animated facial expressions, and gestures to inter- act with users. Inspired by the US military practice of assigning fellow soldiers as partners to provide mutual assistance in both combat and non-combat situations, the name “Battle Buddy” symbolizes this app’s mission.

Battle Buddy’s comprehensive approach aims to establish a suicide pre- vention ecosystem that can be customized to meet the unique needs of individual users. Battle Buddy not only provides valuable in-app health and wellness resources, it also acts as a springboard to real-world support net- works, including friends, family, and various resources provided by the VA and the VCL (Veterans Crisis Line). In the event of a suicidal crisis, Battle Buddy’s primary focus shifts to guiding Veterans through their per- sonalized safety plan, with the goal of creating time and space between suicidal thoughts and actions. By doing so, the app aims to play a cru- cial role in saving lives and providing the necessary support during critical moments.

Figure 1: Screenshots of the battle buddy app.
Original PDF Figure 1:

Our human-centered design model is unique as it incorporates a methodol- ogy borrowed from participatory action-research, Rapid Assessment Process (RAP), within the Information System Research (ISR) framework. The ISR framework has been independently applied in technology development, how- ever, it has not been widely applied to the design of mHealth apps (Schnall et al., 2016). Our team previously defined a novel process for combining RAP with ISR for mHealth development, see (Mozgai et al., 2021). The main focus of this paper is to establish our human-centered design process to identify 1) the mHealth needs of the Veteran user and healthcare providers, 2) their mobile app design preferences in order to maintain user interest, and 3) the barriers that would prohibit uptake and sustained use of this app for a par- ticular population. Though our findings regarding the Battle Buddy app may be specific to the Veteran community, this novel methodology utilizing RAP can be applied across groups of specific end-users with the aim of increas- ing adoption, adherence, and efficacy of mHealth applications. Additionally, we’ll evaluate the acceptability, efficacy, and feasibility of the specific persua- sive strategies employed within the app: (1) personalization (e.g., creating a customized safety plan, (2) self-monitoring (e.g., mood tracking), (3) tun- neling (e.g., guided breathing exercise), (4) suggestion (e.g., app-initiated pushes/prompts), and (5) expertise (e.g., quick connection to suicide hotline).

METHODS

Theoretical Framework The ISR Framework conceptualizes the design process as an embodiment of three closely related cycles of activities (Fig. 2). As applied to the ongoing design of Battle Buddy, the Relevance Cycle helps to bridge the contextual environment of the research project with the design science activities. The Rigor Cycle connects the design science activities with the knowledge base of

Human-Centered Design for a Virtual Human Led mHealth Intervention 101

scientific foundations, experience, and expertise. The central Design Cycle iterates between the core activities of building and evaluating the design artifacts and processes of the research (Hevner, 2007).

Figure 2: The information systems framework adapted from (Hevner, 2007) incorpo- rating rapid assessment process (RAP).
Original PDF Figure 2:

As part of the Design Cycle, our team is engaging in a form of participa- tory action research, Rapid Assessment Process (RAP), a form of intensive, team-based qualitative inquiry using triangulation, iterative data analysis, and additional data collection to get a preliminary understanding of a situa- tion from the insider’s perspective (Beebe, 2001). The strength of this process is that RAP enables a research team to develop a preliminary understanding of a complicated situation in which issues are not yet well-defined – an aim that maps directly onto discovering the needs of a specific group of end-users. RAP shares many characteristics with ethnographic research, however, it is more cost- and time-effective as it utilizes intensive team interaction and rapid cycles of data collection followed by data review and analysis rather than the prolonged fieldwork typically associated with generating insider-levels of understanding.

Rapid assessment is grounded in the qualitative tradition of the early 1970s, specifically taking a systems approach in which all aspects of a local situation are considered. At that time, the acronym for “RAP”also signaled a style of communication that utilized the participant’s vocabulary. The ability of a RAP team to quickly develop a preliminary understanding of a situ- ation is facilitated by having an insider on the research team who has a command of this vocabulary. This insider must be a full team member and must be involved in the planning, data collection, data analysis, and research reporting.

We will briefly describe team activities in the overarching ISR framework.

The Relevance Cycle To understand the desired function and design of the proposed mobile appli- cation for the military community our research team is conducting a series of interviews with various members of the VA, Veterans, and subject matter experts in suicide prevention. Early qualitative review highlights three inter- dependent areas of high priority for US service members and their families: safety planning, barrier reducing interventions, and emotional well-being. Suicide prevention content was then sourced from the VA’s safety planning

manual (Stanley et al., 2022). We are currently conducting a literature review to source evidence-based barrier reducing interventions. Emotional well- being content is sourced from the US Army’s Performance Triad (P3), a guide focused on well-being, and adapted to be interactive (US Army Office of the Surgeon General, 2020).

Table 1. Theoretical frameworks used in the Rigor cycle.

Table available in the original PDF paper.

Description

Theoretical Framework Social Cognitive Theory (SCT) SCT (Bandura, 1986) describes the influence of individual experiences, the actions of others, and environmental factors on individual health behaviors. SCT provides opportunities for social support through instilling expectations, self-efficacy, and using observational learning and other reinforcements to achieve behavior change. In line with SCT of Behavior Change, wearables such as an Apple Watch, Fitbit or Garmin, can facilitate increased self-monitoring, while a Virtual Human coach can provide necessary information and skills for promoting behavior change related to physical, mental, and cognitive well-being. Optimal Matching Theory (OMT)

OMT (Cutrona & Russell, 1990) posits that the effects of social support are enhanced when its provisions are matched with the need for social support. Here, social support can be categorized in the following domains: (1) emotional support, or – “the ability to turn to others for comfort and security during times of stress, leading the person to feel that he or she is cared for by others,” (2) network support, or – “a person feeling part of a group whose members have common interests and concerns,” (3) esteem support, or – “the bolstering of a person’s sense of competence or self-esteem by other people,” (4) tangible aid, or – “concrete instrumental assistance” and, (5) informational support, or – “advice or guidance concerning possible solutions to a problem”. Embodied virtual humans are uniquely positioned to build rapport and provide social support in each of these domains. Functional Triad Fogg introduced the Functional Triad as a framework to simplify the design of persuasive systems (Fogg, 2002). According to this framework, computers can play three main roles. First, they can act as social actors by forming relationships, giving positive feedback, and offering social support. Second, computers as a medium provide experiences that motivate and help people practice behaviors. Lastly, computers can serve as tools by enhancing capabilities, simplifying tasks, guiding users, and performing calculations to encourage desired behaviors. The virtual human in Battle Buddy performs as a social actor designed to excel in each of the categories above. Persuasive Systems Design (PSD)

PSD (Oinas-Kukkonen et al., 2009) is composed of three core elements: 7 postulates behind persuasive systems, 3 ways to analyze the context, and 28 design principles. PSD consists of three key steps; (1), to understand the assumptions behind persuasive systems, (2) analyze the context, and (3) draw out design principles. In particular, the persuasive strategies of (1) personalization (e.g., creating a customized safety plan), (2) self-monitoring (e.g., mood tracking), (3) tunneling (e.g., guided breathing), (4) suggestion (e.g., app-initiated pushes/prompts), and (5) expertise (e.g., quick connection to the crisis line).

Human-Centered Design for a Virtual Human Led mHealth Intervention 103

The Rigor Cycle In the rigor cycle, our team performed a literature review to identify pre- vious technology-based mental-health interventions, including mobile apps, designed for both military and civilian users. In our review of the literature, we did not find any mHealth applications that utilized a virtual human (VH) coach who would guide users through suicide prevention content. However, through our review of the literature exploring engagement and motivation in behavior change we determined cause for the inclusion of a VH coach as well as options for varying the gender and ethnicity of the character, for fur- ther explanation see (Mozgai et al., 2020a). Early signs indicate that virtual humans can provide benefits over human to human interaction related to impression management and reduced perceived bias (Bickmore et al., 2016; Mozgai et al., 2017). Evaluating this novel design aspect of our application and the ability of a VH coach to increase levels of adoption and adherence was therefore deemed high-priority in Design Cycle activities. In addition, we reviewed and selected relevant theoretical foundations on which to guide our content creation processes, see Table 1.

The Design Cycle In the design cycle, our team of clinical psychologists, UX designers, tech- nical artists, and software engineers is building a functional prototype of content, features and functions based on the findings of our literature review. This prototype contains minimal functionality to facilitate quickly receiv- ing user feedback and includes Daily Check-In psychoeducational content (e.g., tactical breathing), a Library of physical and mental health content (e.g., meditation, PHQ-9 assessment), and Safety Plan content (e.g., text a friend, call the Veterans Crisis Line). All content, including VH utterances, were written by our multidisciplinary team and dedicated scriptwriters with both military backgrounds and VH experience. This prototype is currently being adapted with feedback from domain experts and Veterans in order to support suicide prevention and will be the focus of upcoming user testing following the methodology described above.

SYSTEM OVERVIEW This mHealth application has a primary focus on the iPhone and is an extension of a previous virtual human mHealth prototype developed by our team (Mozgai et al., 2020b; Rizzo et al., 2021). Given the initial focus on iPhones and the tight iOS integration, the primary wearable tar- get device is the Apple Watch, which offers a rich multimodal set of sensors. Data will be read directly from the iOS HealthKit API and is processed using our custom application software. This allows us to support any hard- ware device able to write data to HealthKit, including FitBit and Garmin devices. Battle Buddy is a Unity application developed using a custom ver- sion of the Virtual Human Toolkit (Hartholt et al., 2013, 2019, 2022) and RIDE (Hartholt et al., 2021), a rapid prototyping middleware for AI-driven simulations. The VHToolkit incorporates and enables automatic audio-visual sensing, speech recognition, natural language processing, nonverbal behavior

generation, nonverbal behavior realization, text-to-speech generation, and rendering, features which will be added to Battle Buddy through human- centered design processes. Data is leveraged to enable the decision-making algorithms and intervention manager. A subset of the data will be ana- lyzed and evaluated on the client devices to provide users with real-time and actionable feedback (e.g., daily progress towards personal fitness goals, ques- tionnaire results, etc.); the remainder of the data (e.g., UI interaction data, recorded user voice audio) will be collected and post-processed on the server.

FUTURE WORK

When prototype development concludes we anticipate holding our first RAP session as part of the Design Cycle. The primary aim of this first session will be to determine alignment between our initial prototype and user-identified 1) mHealth needs, 2) mobile app design preferences (including VH elements), and 3) barriers to adoption and adherence. Successful RAP sessions depend on teamwork. Multidisciplinary research teams, comprised of both insiders and outsiders, have been shown to increase sensitivity to insider categories and definitions (Beebe, 2001). Our multidisciplinary team will consist of both insiders (e.g., US military Veterans), and outsiders (e.g., Psychologists, Research Assistants, Software Developers, Content Creators, and Artists) who will contribute to study design, data collection, data analysis, and prototype refinement.

RAP is an iterative process that explicitly divides research time between blocks used for collecting information and blocks when the RAP team con- ducts data analysis and considers changes for successive rounds of data collection. Following Miles’ and Huberman’s data analysis model our team will 1) code the data, 2) visualize the data, and 3) draw conclusion on the data (Miles & Huberman, 1994). The preliminary findings of our first ses- sion of RAP will be utilized by the team to inform future rounds of RAP and refine app design.

ACKNOWLEDGMENT

This work was sponsored by the US Army under contract W911NF-14- D-0005 and the VA Mission Daybreak Challenge, in collaboration with SoldierStrong. The content of the information does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.

REFERENCES

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall. Beebe, J. (2001). Rapid assessment process: An introduction. Rowman Altamira. Bickmore, T. W., Utami, D., Matsuyama, R., & Paasche-Orlow, M. K. (2016).

Improving Access to Online Health Information With Conversational Agents: A Randomized Controlled Experiment. Journal of Medical Internet Research, 18(1), e1. https://doi.org/10.2196/jmir.5239

Human-Centered Design for a Virtual Human Led mHealth Intervention 105

Cutrona, C., & Russell, D. (1990). Type of social support and specific stress: Toward a theory of optimal matching. https://psycnet.apa.org/record/1990-97699-013 Fogg, B. J. (2002). Persuasive technology: using computers to change what we think

and do. Ubiquity, 2002(December), 2. https://doi.org/10.1145/764008.763957 Hartholt, A., Fast, E., Li, Z., Kim, K., Leeds, A., & Mozgai, S. (2022). Rearchitecting the virtual human toolkit: Towards an interoperable platform for embodied conversational agent research and development. IVA 2022 - Proceed- ings of the 22nd ACM International Conference on Intelligent Virtual Agents. https://doi.org/10.1145/3514197.3549671 Hartholt, A., Fast, E., Reilly, A., Whitcup, W., Liewer, M., & Mozgai, S. (2019).

Ubiquitous virtual humans: A multi-platform framework for embodied ai agents in xr. Proceedings - 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2019, 308–312. https://doi.org/10.1109/AIVR46125. 2019.00072 Hartholt, A., McCullough, K., Fast, E., Reilly, A., Leeds, A., Mozgai, S., Ustun, V., Gordon, A. S., & McGroarty, C. (2021). Rapid Prototyping for Simulation and Training with the Rapid Integration & Development Environment (RIDE). Interservice/Industry Training, Simulation and Education Conference (I/ITSEC). https://www.researchgate.net/publication/365368647 Hartholt, A., Traum, D., Marsella, S. C., Shapiro, A., Stratou, G., Leuski, A.,

Morency, L.-P., & Gratch, J. (2013). All Together Now: Introducing the Virtual Human Toolkit. Intelligent Virtual Agents; IVA 2013, 368–381. https://doi.org/ 10.1007/978-3-642-40415-3_33 Hevner, A. (2007). A three cycle view of design science research. Scandinavian Journal of Information. Miles, M., & Huberman, A. (1994). Qualitative data analysis: An expanded

sourcebook. Mozgai, S., Femminella, B., Hartholt, A., & Rizzo, A. (2021). User-Centered Design Model for Mobile Health (mHealth) Applications: A Military Case Study in Rapid Assessment Process (RAP). Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3411763.3443450 Mozgai, S., Hartholt, A., & Rizzo, A. (2020a). The Passive Sensing Agent: A Mul-

timodal Adaptive mHealth Application. 2020 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2020. https://doi.org/10.1109/PerComWorkshops48775.2020.9156177 Mozgai, S., Hartholt, A., & Rizzo, A. (2020b). An adaptive agent-based interface for personalized health interventions. International Conference on Intelligent User Interfaces, Proceedings IUI, 37(111), 118–119. https://doi.org/10.1145/3379336. 3381467 Mozgai, S., Lucas, G., Gratch, J., Mozgai, S., Lucas, G., & Gratch, J. (2017). To

Tell the Truth: Virtual Agents and Morning Morality. In Proceedings of the 17th International Conference on Intelligent Virtual Agents (pp. 283–286). Springer International Publishing. https://doi.org/10.1007/978-3-319-67401-8_37 Oinas-Kukkonen, H., Systems, M. H. I., & 2009, U. (2009). Persuasive systems design: Key issues, process model, and system features. Aisel. Aisnet. Org, 24, 485–500. Prevention, O. of M. H. and S. (2022). National Veteran Suicide Prevention Annual

Report. Rizzo, A. S., Hartholt, A., & Mozgai, S. (2021). From Combat to COVID-19– Managing the Impact of Trauma Using Virtual Reality. Journal of Technology in Human Services, 39(3), 314–347. https://doi.org/10.1080/15228835.2021. 1915931

Schnall, R., Rojas, M., Bakken, S., Brown, W., Carballo-Dieguez, A., Carry, M., Gelaude, D., Mosley, J. P., & Travers, J. (2016). A user-centered model for design- ing consumer mobile health (mHealth) applications (apps). Journal of Biomedical Informatics, 60, 243–251. https://doi.org/10.1016/J. JBI.2016.02.002 Stanley, B., Brown, G. K., MacRae, F., Rotolo, C. A., Hughes, G., Mina, L., & Barry,

C. N. (2022). VA safety planning intervention manual. US Department of Veterans Affairs. Tam-Seto, L., Wood, V., Linden, B., Mhealth, H. S.-, & 2018, U. (2018). A scoping review of mental health mobile apps for use by the military community. Ncbi. Nlm. Nih. Gov. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6327160/ Torous, J., Psychiatry, L. R.-J., & 2017, U. (2017). Needed innovation in digital

health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry, 74(5), 437–438. https://jamanetwork.com/journals/jamapsych iatry/article-abstract/2616170 Trifan, A., Oliveira, M., UHealth, J. O.-J. mHealth and, & 2019, U. (2019). Passive sensing of health outcomes through smartphones: systematic review of current solutions and possible limitations. JMIR MHealth and UHealth, 7(8). https://mh ealth.jmir.org/2019/8/e12649/ US Army Office of the Surgeon General. (2020). Performance Triad (P3). https://p3.

amedd.army.mil/about Yardley, L., Morrison, L., Bradbury, K., & Muller, I. (2015). The person-based approach to intervention development: Application to digital health-related behavior change interventions. Journal of Medical Internet Research, 17(1), e30. https://doi.org/10.2196/jmir.4055

Cover of the proceedings volume Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-88-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →