Olarinde Farayola, Dastyni Loksa, Jinjuan Feng (2024). Unraveling Interaction Challenges for Deaf and Hard-of-Hearing Users: An Exploration of Digital Content and Interfaces Accessibility. In: Accessibility, Assistive Technology and Digital Environments. AHFE Open Access, vol. 121, pp. 152–160. https://doi.org/10.54941/ahfe1004623
Efficient engagement with ubiquitous digital content and interfaces presupposes a profound level of reading comprehension, a potential impediment for users within the deaf and hard-of-hearing community. Existing literature demonstrates a correlation between diminished reading comprehension and deafness [1, 3], accentuating the formidable barriers faced by this demographic in accessing and understanding digital content and navigating an increasingly digital world. Despite decades of efforts to enhance digital accessibility for all users, limited research has focused on the specific challenges confronting deaf and hard-of-hearing individuals, particularly those not adept with technology [2].This study aims to address this gap by investigating the daily interaction challenges faced by deaf and hard-of-hearing users with digital content and interfaces. Conducting a focus group with four deaf participants, we chose this method to facilitate rich, in-depth discussions in American Sign Language (ASL), a mode of communication familiar to the participants. We selected participants with educational levels from high school to those who had attained an Associate's degree seeking to get a range of possible challenges and investigating if their level of education might impact the challenges they face.Qualitative analysis of the focus group's dialogue unveiled insights into the technological struggles experienced by participants, coupled with their expressed needs for technological support. Three salient themes emerged: 1. Difficulties comprehending textual information across various tasks (email, online search, reading news, text messages, etc.), as unanimously reported by all participants; 2. Challenges associated with writing text and text entry, universally confirmed by all participants; and 3. Challenges in content and interface comprehension, influencing the installation, navigation, and utilization of various technologies, universally acknowledged by all participants.In this paper, we discuss the implications of these findings for researchers, technology developers, and digital content providers alike. We further explore avenues for future research along with potential technological enhancements aimed at mitigating the challenges confronting the deaf and hard-of-hearing population. The insights derived from this study are positioned to guide digital content developers, policymakers, and educators in effecting essential changes to amplify the accessibility of digital content and interfaces for deaf and hard-of-hearing users, both within the United States and on a global scale. References[1]M. Marschark and P. C. Hauser, Deaf cognition: foundations and outcomes. Oxford ; New York: Oxford University Press, 2008.[2]M. Maiorana-Basas and C. M. Pagliaro, “Technology Use Among Adults Who Are Deaf and Hard of Hearing: A National Survey,” Journal of Deaf Studies and Deaf Education, vol. 19, no. 3, pp. 400–410, Mar. 2014, doi: https://doi.org/10.1093/deafed/enu005.[3]E. Toofaninejad, E. Zaraii Zavaraki, S. Dawson, O. Poquet, and P. Sharifi Daramadi, “Social media use for deaf and hard of hearing students in educational settings: a systematic review of literature,” Deafness & Education International, vol. 19, no. 3–4, pp. 144–161, Oct. 2017, doi: https://doi.org/10.1080/14643154.2017.1411874.
The population of deaf and hard-of-hearing (DHH) people in the United States is rapidly increasing due to a range of factors, including but not lim- ited to aging, genetic predisposition, medication side effects, exposure to excessively loud noise, and injuries (Wirth, 2022). Hearing loss affects about 60.7 million people (about twice the population of Texas) aged 12 and older in America, and about 15.5% (approximately 44.1 million) of American adults aged 20 and older experience some degree of hearing loss (Hernan- dez, 2023). The World Health Organization estimates that globally, around 466 million people have hearing loss, constituting about 6.1% of the world’s population. Alarmingly, this number is projected to rise to 630 million by 2030 and exceed 2 billion by 2050 (WHO, 2023). Research has established a clear correlation between deafness and reading comprehension. Both in the United States and globally, there is well-documented evidence indicating
Unraveling Interaction Challenges for Deaf and Hard-of-Hearing Users 153
that deafness significantly impedes literacy (Hlatywayo and Zano Muranda, 2015). This interference extends to challenges in reading, writing, compre- hension, recalling, recognizing, and task performance, with the extent of impact varying based on factors such as educational experiences, family cir- cumstances, age of onset of hearing loss, and the degree of hearing loss in individuals.
RELATED RESEARCH Deafness is a sensory disability characterized by a partial or complete inability to hear sounds at a typical or expected level. It can be mild, moderate, mod- erately severe, severe, or profound, and can affect one or both ears (Vidhya, 2020). Sensorineural hearing loss is one of the most prevalent hearing losses, which is caused by “damage to the auditory nerve or sensory cells in the inner ear” (Everett, 2023).
Deafness and Literacy Deafness can affect reading comprehension due to the absence of auditory input, which is crucial for language development. The extent of hearing loss can vary greatly among DHH users, impacting their ability to perceive and comprehend auditory information. Research has shown that deaf individuals may face challenges in acquiring language skills, especially in understand- ing syntax, semantics, and pragmatics (Kelly and Berent, 2021). Research like those conducted by Marschark and Hauser delves into the relationship between deafness and language acquisition, highlighting how it can affect various aspects of reading comprehension. They discuss how limitations in language proficiency can hinder the ability to grasp written text effectively (Marschark, et al. 2008). Studies have demonstrated how deaf students lag behind their hearing peers in written composition and spelling acquisition, impacting their education, communication with people, access to decent employment, and becoming involved citizens (Daigle and Berthiaume, 2020).
Digital Technology and Design for DHH Users The association between cognition and navigating digital content has been confirmed, emphasizing the importance of cognitive processes in designing interactive systems for DHH users (Fajardo, et al. 2008). Advancements in Human-Computer Interaction (HCI) have increasingly focused on making digital interfaces more accessible, usable, convenient, and efficient for deaf users. For instance, studies by (Dermawi, et al. 2018) emphasize the usabil- ity of digital interfaces for the deaf community, highlighting the significance of involving DHH users in developing applications. Specific interface types such as graphical user interfaces (GUIs) have been examined in making infor- mation more accessible to deaf individuals. These interfaces employ visual elements, icons, and design principles that cater to users reliant on visual cues and textual information. In a study conducted by Potter, et al., they observed deaf children’s interaction with technology and stressed the importance of involving DHH users in designing technologies to contain their character- istics such as reduced literacy, delayed language, limited communication,
slower cognition, and nervousness (Potter, et al. 2014). The acceptance and efficacy of digital interfaces within the Deaf community are inherently tied to the embrace of Deaf culture. Bauman and Murray challenge deficit-based perceptions of deafness and promote the celebration of Deaf culture’s unique strengths (Bauman and Murray, 2014). Incorporating these cultural values into interface design fosters greater acceptance and usability within the Deaf community.
The challenges deaf users encounter in reading within interactive systems have been a focal point in research. Literacy challenges among Deaf students were investigated, highlighting the impact of language barriers, limited access to early intervention, and varying educational approaches (Easterbrooks and Beal-Alvarez, 2013). An emphasis has been placed on the importance of social media platforms as “integral to modern entertainment, civic engagement, news dissemination, and interpersonal communication” (Mack, et al. 2020). However, the results of the study conclude that deaf participants expressed challenges and frustrations with text-based, uncaptioned content, and audio forms of communication. Researchers like Fajardo, Cañas, Salmerón, and Abascal carried out a study that considers cognitive aspects of prelingual deaf users in web information search, especially information structure that is commensurate with their mental abilities in accessing web information. They recommend aids to support the comprehension of the content and to use information scents, which they describe as “interpretations of the relevance of local cues such as textual links and images” (Fajardo, et al. 2009).
Researchers have attempted to accommodate these diverse reading levels in their research focusing on text simplification that targets congenitally deaf individuals. In their research approach, they collected readability assessment data from teachers of deaf students by asking them to compare the readabil- ity of a given sentence with its paraphrases since they knew the language proficiency of their deaf students and possess the needed skills in paraphras- ing sentences that resonate with deaf students’ reading and writing levels (Inui, et al. 2024). In a similar study to assist deaf and hard-of-hearing users in text simplification, unlike Inui, et al.’s data collection approach, a researcher, Alonzo conducted a survey and interview with deaf and hard- of-hearing adults directly to assess their reading experiences in technologies. Their responses reveal a strong interest in Automatic Text Simplification- based reading assistance tools, which reduce the linguistic complexity of texts while preserving the original forms and meanings (Alonzo, 2022). Similarly, researcher Okuyama investigated deaf adolescents’ texting for communica- tion and found that their text composition was a reflection or structure of how they communicate in American Sign Language (ASL), which has “several word orders: Subject-Verb-Object or Subject-Verb order and Time-Subject- Verb-Object or Time-Subject-Verb word order” (Okuyama, 2013). A study by (Power, et al. 2006) citing (Pilling, et al. 2007), surveyed the use of different forms of text communication (SMS, text relay, TTY, e-mail, IM, and fax) among deaf users in Australia and found SMS, fax, and IM to be the “most frequently used methods of communication among users who were prelingually profoundly or severely deaf than for the whole group with
Unraveling Interaction Challenges for Deaf and Hard-of-Hearing Users 155
severe or profound deafness”. However, Pilling, et al. did not directly exam- ine the impact of deafness on usability, accessibility, performance, and user satisfaction.
METHODOLOGY
Research Design and Data Collection This study explores the interactions between deaf users and digital interfaces and content through a focus group. We conducted the focus group to facil- itate discussions on user experiences within the DHH community. The first author, who is post-lingually deaf, moderated the focus group in ASL and was assisted by 2 professional ASL interpreters and two assistant moderators who were neither hard of hearing nor proficient in ASL. The focus group was con- ducted in person at Towson University’s main campus and was 2 hours and 30 minutes in length. During the focus group, we projected PowerPoint slides displaying the current discussion topic with supporting images to help par- ticipants understand the topic and reduce confusion. The discussion topics were phrased as questions focused on participants’ challenges and user expe- riences with digital content and interfaces.The focus group was audio and video recorded on two separate devices. Additionally, the two assistant mod- erators took notes on participant responses during the focus group. Through thematic analysis, we collected and analyzed data to generate insights and conceptual understanding.
Participants We recruited four deaf and hard-of-hearing (DHH) participants for the focus group. Recruitment involved reaching out to them through flyers and per- sonal contacts with deaf associations, clubs, and churches in Baltimore City, Maryland. Participants in this study were all deaf and 18 years or older, with limited to moderate education/reading comprehension, and primarily use ASL as a preferred communication method. Participant 1 (P1) is a deaf male, aged 46, received an AA degree, and used ASL as a medium of com- munication. Participant 2 (P2) is a deaf female, aged 43, a fresh student in community college, and who used ASL as her preferred language. Participant 3 (P3) is a deaf male, aged 64, a High School graduate, and used ASL to communicate. Participant 4 (P4) is a deaf male, aged 34, a High School grad- uate with no desire to further studies, and used ASL as a preferred medium of communication. All the participants came with their cell phones used for text and video communication. Each participant received $50 in compensation for their participation after completing the focus group.
Data Collection The study’s purpose was to investigate the experiences of people who are DHH when interacting with the current digital content and interfaces. After obtaining written consent from the participants, we presented 8 topics that were related to the research questions. Participants discussed among them- selves and with the moderators using American Sign Language as a mode of
communication to express their concerns, opinions, views, and suggestions, while the investigators listened attentively, asked follow-up and clarification questions, and took notes.
Thematic Analysis The data collected was audio-visual (voice and ASL) recordings and hand- written or typed notes taken by investigators during the focus group. To analyze the data, we conducted a Thematic Analysis (Braun and Clarke, 2006) independently reviewing the data before discussing and agreeing on a list of themes and patterns pertinent to our research questions. We then analyzed the recordings to further identify and categorize each occurrence of the selected patterns, ensuring their validity. This analysis was done by both hearing and deaf authors to control for possible interpreter translation bias or misinterpretations of participants’ responses.
FINDINGS Three salient themes emerged from the focus group data analysis, aligning with the interaction challenges in the digital content and interfaces among deaf and hard-of-hearing users. We found 42 total instances of relevant statements and categorized them into the following themes.
Theme 1: Difficulties comprehending textual information across various tasks. For example, all participants expressed difficulty reading and com- prehending English-text-based digital content and interfaces. Participants stated a preference for an alternative simplified text by minimizing the com- plexity associated with text-based content and interfaces to commensurate with their reading levels. As visual readers, participants also discussed read- ing and comprehending text-based interfaces in sign language as a form of accessibility.
Table 1. Examples of responses in Theme 1.
Table available in the original PDF paper.
Participant Example Responses/Excerpts P1 “ASL is our own language, and we are trying to understand something” P2 “I don’t always understand hearing people’s texts” P3 “Anything that has to be read in English should have interpretations in ASL” P4 “Reading the words can be difficult. Sometimes I have to go look up a word to see what it means”
Theme 2: Writing text and text entry: This theme was derived from excerpts regarding participants’ writing levels. In the focus group discussion, participants mentioned difficulty in expressively and/or accurately writing textual input when using digital content and interfaces. Participants strongly desire to make their text input readable and comprehensive to the machine or systems or other people, especially hearing people, when seeking informa- tion online or communicating with others, considering their limited literacy.
Unraveling Interaction Challenges for Deaf and Hard-of-Hearing Users 157
In addition, participants also wish to input information using sign language and have it recognized by the interactive systems.
Table 2. Examples of responses in Theme 2.
Table available in the original PDF paper.
Participant Example Responses/Excerpts P1 “[My difficulty] is using words, trying to type. There is no sign language to help me understand if what I’ve typed is wrong” P2 “When texting, I have to think about how to text so the
[hearing people] understand. Sometimes I have to do it again and again” P3 “I use Grammarly that helps with writing English and translating, it is frustrating but helps a lot.” P4 “I struggle with writing and have to explain everything.”
Theme 3: Content and interface comprehension: All participants reported unfriendly interface designs and navigation. In our discussion, cognitive challenges in navigating interfaces and finding information were mentioned unanimously. For example, participants agreed that using Google was tough because there was no backup option to further explain things to them. All participants wished to have chat boxes that would pop up with sign language to help clarify information for them or ask them how the chat window could help them.
Table 3. Examples of responses in Theme 3.
Table available in the original PDF paper.
Participant Example Responses/Excerpts P1 “I need something to pop up with sign language that could help us” P2 “Captioning on [MS] teams can sometimes be wrong and doesn’t make sense. Sometimes I don’t understand, but sometimes it is just wrong.” P3 “Using Google is tough, really tough. I wish there was a backup option like something that would pop up to further explain things for me.” P4 “I would love to have Apple have more interpreter picture-in-picture"
DISCUSSION
In this study, we conducted a focus group to investigate DHH users’ expe- riences with digital content and interfaces occasioned by limited literacy. To ensure authentic responses from our participants, our study was con- ducted using ASL, our participants’ preferred mode of communication. This is a departure from previous studies by (Maiorana-Basas & Pagliaro, 2014; Pilling et al. 2007; Okuyama, 2013; Inui et al. 2024) that adopted written questionnaires and surveys to gather data from DHH respondents despite their limited literacy. We believe that using an in-person focus group enabled us to observe participants’ struggles in responding to topics and provide additional context or clarification in ASL, facilitating full and meaningful responses.
Our findings highlight the role of deafness in limiting the effectiveness and efficiency of digital content and interface use. Our participants reported
facing struggles with comprehending textual information or output, writ- ing and/or entering textual information or input, and navigating content and interface across various tasks. Our findings follow from prior work: Theme 1 supports studies by Daigle, et al. on deficiency in written compo- sition and spelling acquisition among deaf individuals (Daigle, et al. 2020); Theme 2 emerged as a ubiquitous theme from all participants, corroborating (Marschark and Hauser, 2008) on how limitations in language proficiency can hinder the ability to grasp written text effectively, and a study by (Inui et al. 2024) on text simplification (TS) for the deaf users; and Theme 3 aligns with Fajardo et al.’s work (Fajardo, et al. 2019) which supports comprehension of the content and to use information scents.
Our findings suggest a few possible solutions and show the importance of including the experiences of DHH people when designing digital content and interfaces. As the disability saying goes: “Nothing about us without us” (Yeo & Moore, 2003). Participants recommended incorporating sign language into textual information in both input and output. Sign language interpreters in picture-in-picture can be incorporated into interactive systems to aid deaf users in navigating content and interfaces with few clicks. They also suggested that interactive systems be designed to accommodate DHH users’ own com- positional written English (“Deaf English”) which is different from standard written English. However, incorporating sign language into digital content and interfaces has faced challenges due to the lack of variety in the dataset and limited availability of the dataset. According to a researcher, Rogers, Sign Language technologies are “globally not robust because the features only work for a tiny set of fixed commands and recognizing or translating sign language messages is performed unnaturally”. Bragg, et al. suggest that Deaf studies be included to understand the users and build efficient Sign Language technology (Bragg, et al. 2019).
The implications for the themes expressed in our findings are clear: Design- ers and developers of interfaces as well as digital content providers should recognize and address the needs of DHH users. It is important to help design- ers create better experiences, improve digital literacy for DHH users, and work toward improving U.S laws, such as World Wide Web Consortium (W3C) Web Content Accessibility Guidelines 2.1 (WCAG 2.1), Title III of the Americans with Disabilities Act, and Section 508 of the Rehabilitation Act that protects accessibility requirements.While our participants demon- strated a considerable level of agreement in their responses, it is important to acknowledge the limitation of the small sample size.Our participants shared regional context could influence the results, potentially missing the diversity of experiences across different geographical or cultural settings. Furthermore, the study did not explore the many possible digital contexts that may be used by deaf individuals, potentially overlooking nuances of specific technologies. Therefore, the findings of the study should be interpreted with caution, and further studies are needed to generalize the findings to the broader DHH population.
Unraveling Interaction Challenges for Deaf and Hard-of-Hearing Users 159
CONCLUSION
Over the last few years, digital literacy skills have become one of the most important assets for individuals. Not only are they important for individuals in digital spaces, but also important for users of organizational systems such as education, healthcare, communication, economics, industry, and other sec- tors. Nevertheless, not many studies have explored the interaction challenges DHH users face in digital spaces. The view that deafness is positively cor- related with limited digital literacy is consistent with this study. Deafness impedes reading comprehension with which DHH users can effectively and efficiently interact with digital content and interfaces depriving DHH users of accessibility and user satisfaction. Human-computer interaction plays a crucial role in reducing these challenges and increasing efficiency by consid- ering the needs of human users in the design process. Involving DHH users in this study has facilitated an understanding of their pain points, preferences, and needs. From the salient themes that emerged, Sign Language Recognition (SLR) and Text Simplification (TS) are key to accessibility and usability for DHH users in an interactive system.
REFERENCES
Alonzo, O. (2022). The use of automatic text simplification to provide reading assistance to deaf and hard-of-hearing individuals in computing fields. ACM SIGACCESS Accessibility and Computing, 132, 1–1. https://doi.org/10.1145/3523265.3523268 Bauman, H.-D. L., & Murray, J. J. (2014). Deaf Gain: Raising the Stakes for Human Diversity. In Google Books. U of Minnesota Press. Bragg, D., Koller, O., Bellard, M., Berke, L., Boudreault, P., Braffort, A., Caselli, N.,
Huenerfauth, M., Kacorri, H., Verhoef, T., Vogler, C., & Ringel Morris, M. (2019). Sign Language Recognition, Generation, and Translation. The 21st International ACM SIGACCESS Conference on Computers and Accessibility. Braun, V., & Clarke, V. (2006). Using Thematic Analysis in Psychology. Qualitative Research in Psychology, 3(2), 77–101. Tandf online. Daigle, D., Berthiaume, R., Costerg, A., & Plisson, A. (2020). What Do Spelling
Errors Tell Us about Deaf Learners of French? The Journal of Deaf Studies and Deaf Education. Dermawi, R., Tolle, H., & Aknuranda, I. (2018). Design and Usability Evaluation of Communication Board for Deaf People with User-Centered Design Approach. International Journal of Interactive Mobile Technologies (IJIM), 12(2), 197. https://doi.org/10.3991/ijim.v12i2.8100 Easterbrooks, S. R., & Beal-Alvarez, J. (2013). Literacy instruction for students who
are deaf and hard of hearing. Oxford University Press. Everett, C. (2023). Hearing Loss Is More Common Than Dia- betes. Why Aren’t We Addressing It? National Council on Aging. https://www.ncoa.org/adviser/hearing-aids/hearing-loss-america/#footnote-10 Fajardo, I., Cañas, J. J., Salmerón, L., & Abascal, J. (2009). Informa- tion structure and practice as facilitators of deaf users’ navigation in tex- tual websites. Behaviour & Information Technology, 28(1), 87–97. https://doi.org/10.1080/01449290801988290 Fajardo, I., Abascal, J., & Cañas, J. J. (2008). Bridging the digital divide for deaf signer users. CiteSeer X (the Pennsylvania State University).
Glasser, A., Mande, V., & Huenerfauth, M. (2020). Accessibility for Deaf and Hard of Hearing Users. Proceedings of the 2nd Conference on Conversational User Interfaces. https://doi.org/10.1145/3405755.3406158 Hernandez, N. (2023, November 8). Hearing Loss Statistics 2023: Numbers We Can’t Ignore. National Council on Aging. https://www.ncoa.org/adviser/hearing- aids/hearing-loss-statistics/ Hlatywayo, L. and Muranda, A. Z., (2015). The impact of hearing loss on literacy
development: The role of the home and school. International Journal of Novel Research in Education and Learning, 2(1), pp. 1–9. Inui, K., Fujita, A., Takahashi, T., Iida, R. and Iwakura, T., 2003, July. Text simplification for reading assistance: A project note. In Proceedings of the second international workshop on Paraphrasing (pp. 9–16). Kelly, R. R., & Berent, G. P. (2011). Semantic and Pragmatic Factors Influencing Deaf
and Hearing Students’ Comprehension of English Sentences Containing Numeral Quantifiers. Journal of Deaf Studies and Deaf Education, 16(4), 419–436. Mack, K., Bragg, D., Morris, M. R., Bos, M. W., Albi, I., & Monroy- Hernández, A. (2020). Social App Accessibility for Deaf Signers. Pro- ceedings of the ACM on Human-Computer Interaction, 4(CSCW2), 1–31. https://doi.org/10.1145/3415196 Maiorana-Basas, M., & Pagliaro, C. M. (2014). Technology Use Among Adults Who
Are Deaf and Hard of Hearing: A National Survey. Journal of Deaf Studies and Deaf Education, 19(3), 400–410. Marschark, M., Hauser, P. C., & Netlibrary, I. (2008). Deaf cognition : foundations and outcomes. Oxford University Press. Mayer, C., & Trezek, B. J. (2023). Communication, Language, and Modality in the
Education of Deaf Students. Education Sciences, 13(10), 1033–1033. Mayer, C., & Trezek, B. (2019). Writing and Deafness: State of the Evidence and Implications for Research and Practice. Education Sciences, 9(3), 185. Okuyama, Y. (2013). A case study of US deaf teens’ text messaging: Their innovations
and adoption of textisms. New Media & Society, 15(8), 1224–1240. Pilling, D., & Barrett, P. (2007). Text Communication Preferences of Deaf People in the United Kingdom. Journal of Deaf Studies and Deaf Education, 13(1), 92–103. Potter, L. E., Korte, J., & Nielsen, S. (2014). Design with the deaf. Proceedings of
the 2014 Conference on Interaction Design and Children. Power, M. R., Power, D., & Horstmanshof, L. (2006). Deaf People Communicating via SMS, TTY, Relay Service, Fax, and Computers in Aus- tralia. Journal of Deaf Studies and Deaf Education, 12(1), 80–92. https://doi.org/10.1093/deafed/enl016 Vidhya, V. (2020). Hearing Impairment [Review of Hearing Impair- ment]. Inclusive Education, 978-81-945642-8-7, 275–287. https://d1wqtxts1xzle7.cloudfront.net/67231573/Inclusive_Education- libre.pdf?1620313585 Wirth, J. (2022, December 19). Deafness And Hearing Loss Statistics. Forbes Health.
https://www.forbes.com/health/hearing-aids/deafness-statistics/ World Health Organization. (2023, February 27). Deafness and hearing loss. WHO; World Health Organization: WHO. https://www.who.int/news-room/fact- sheets/detail/deafness-and-hearing-loss Yeo, R., & Moore, K. (2003). Including Disabled People in Poverty Reduction Work: “Nothing About Us, Without Us.” World Development, 31(3), 571–590.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Accessibility, Assistive Technology and Digital Environments
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-97-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional