Alexis Polanco Jr, Tsailu Liu (2023). Multidisciplinary Framework for Creating the Next-generation of Human-centered Design Guidelines. In: Human-Centered Design and User Experience. AHFE Open Access, vol. 114, pp. 84–92. https://doi.org/10.54941/ahfe1004225
Human-centered design for students with disabilities is an inherently multidisciplinary endeavor. It requires input from practitioners in the fields of user experience (UX), instructional design, accessibility, software engineering, and more. To that end, this paper seeks to reveal the perspectives of practitioners who have experience across these various disciplines, especially as they pertain to Deaf and hard of hearing (DHH) students.In a previous study, the author asked practitioners to describe the design guidelines they use in their professions. To contextualize the concept of guidelines, the research participants were introduced to North Carolina State University’s Principles of Universal Design and Jakob Nielsen’s 10 Usability Heuristics. The outcome of that study revealed a wide spectrum of guidelines, with little commonality except for the Web Content Accessibility Guidelines and Universal Design.This paper builds off that prior research by interviewing the same practitioners with a renewed focus on the definition of guidelines. In particular: 1) How do these practitioners define “guidelines”, and do these definitions vary between the different disciplines of user experience, accessibility, etc.?2) Do the practitioners’ definitions of “guidelines” change as the end-user becomes more specifically designed? For example, do practitioners expect more, less, or something different when designing for hearing students vs DHH students?3) If these practitioners were empowered to create a theoretical set of design guidelines for DHH middle school-aged students, what process would they follow? The results of this new study demonstrate an awareness in the research participants that experts from different disciplines may need to be engaged in the process of creating design guidelines. More specifically, when asked to create DHH guidelines, the SMEs were often quick to say that they needed to collaborate with individuals who were embedded in the DHH community, i.e., individuals who are Deaf or hard of hearing, or individuals who have acquired second-hand knowledge through being immersed in Deaf culture.Surprisingly, when interviewing these practitioners for a second time, designers were not always explicitly stated as integral to the creation of design guidelines for DHH middle school students. A few reasons emerged for why this is the case: A) On multiple occasions, the research participants lamented that design education today often fails to prepare new designers in the space of accessibility. B) In another instance, a senior software architect felt that engaging designers would be detrimental to the guideline creation process. He would later amend his response to say that he would include a designer only if they had a sufficient background in accessibility. C) In a different instance, a designer took designer participation for granted and assumed that designers would obviously be part of the design guideline process. When asked to clarify why he did not include designers in his process when first asked, he said that his ego led him to assume that he would naturally be part of the process.While both this paper and the previous paper focus on the DHH student experience, the overall goal of this research is to create a reusable framework that will allow subject matter experts (SMEs) to examine the guidelines they use today, and to learn how to create the next generation of design guidelines that will rise to the challenge of human-centered design becoming increasingly multidisciplinary.
Multidisciplinary Framework for Creating the Next-Generation 85
INTRODUCTION
The author’s initial research goal was the creation of guidelines for practi- tioners who were involved in the design of products for Deaf and hard of hearing (DHH) children. This goal was formed in response to the seemingly non-existent research into Deaf and hard of hearing children, despite marked increases in the Deaf child population every year (Pizzo & Chilvers, 2016). To provide context on this growing population group, between 2005 and 2020, the United States Center for Disease Control and Prevention reported a 239% increase in the number babies born Deaf or hard of hearing who were identified early (CDC, 2022).
To accelerate the creation and release of much-needed DHH child design guidelines to the public, the author sought to narrow his focus to the field of Deaf and hard of hearing assessment design. The resulted in the paper, “Uncovering an Inclusion Gap in the Design of Digital Assessments for Mid- dle school-aged Deaf and Hard of Hearing Students in the United States” (Polanco Jr & Liu, 2023).
That paper revealed that leading testing institutions are prone to pretesting their assessment items without including students with disabilities for reasons including, but not limited to, the fact that these populations often fall below the statistical thresholds that would make them candidates for pretesting can- didate selection, especially when using oft-quoted rules of thumbs for these activities (ETS, 2020) (Pearson, 2018). For example, it’s known that inter- viewing at least five people reveals most usability problems. (Nielsen, 2012) Given that the number of students in the United States with disabilities gen- erally fall below 20% of the total U.S. student population, it’s very likely students with disabilities will he wholly omitted from the design process, unless they are explicitly sought out (NCES, 2020).
Choosing to focus on the Deaf and hard of hearing student experience, the author interviewed 13 practitioners in the fields of user experience (UX), accessibility, and education and had them self-identify their roles, their work process, their familiarity with both DHH (Deaf and hard of hearing) users, and their professional opinion on the difference between designing for adults and designing for children.
This resulted in the paper “UX Design for children: How is it done now? Should it change?”, which will hopefully be published early next year. (Polanco Jr & Liu, n.d.) At a high-level, three outcomes emerged from that study:
1. Designers questioned how relevant their experience is to subject matter experts in child computer interaction; subject matter experts doubted the value of their expertise in the design of products for children. 2. The design process for practitioners who worked with children did not vary significantly from those who only had experience designing for adults. 3. The primary difference between products for children and adults is the age-level appropriateness of the content. As such, it is of the utmost importance that at least one practitioner in the child-product
86 Polanco Jr and Liu
creation process includes a content specialist who understands the dif- ference between content for adults and content for children of different age-levels.
STUDY DESIGN In the author’s previous study, “UX Design for children”, 13 practitioners in the space of user experience, accessibility, and education were interviewed for 45-to-60 minutes on the topic of digital assessment for Deaf and Hard of hearing students. This study took place over two years and used social media, email, and word-of-mouth recruitment techniques. Recruitment proved espe- cially difficult, with a known factor being that participants were not paid for volunteering their time. Financial incentives would have resulted in signifi- cant increases in research participation, but the author’s research committee was uncertain of the effects of incentives on the participants’ responses (Abdelazeem et al., 2022).
The questions asked were as follows: Interviewee Classification
What is your job title?
How do you define your field of expertise?
Who do you design for?
– If not children, do your products get used by children? Why or why not? – If age is not specified, do you think middle school-aged students use your products?
Work process
What design principles are you aware of and use in your profession?
How would you describe your daily tasks?
Does your work affect a deaf or hard of hearing audience?
Child-computer Interaction
Does your work process vary based on whether your target is an adult or child?
– If yes, how does your design process vary from adult-targeted products to child-targeted products? – If no, why doesn’t your design process vary from adult-targeted products to child-targeted products?
What would you change about your current process, if anything, to suit child audiences?
Deaf or Hard of Hearing (DHH)
Does your work process vary based on whether your target audience is deaf or hard of hearing? If yes, how? If no, why not?
Multidisciplinary Framework for Creating the Next-Generation 87
Evaluation of the interview transcripts revealed the possibility that one of the interview questions may have misled the participants. Notably, one ques- tion asked what design principles the practitioners were aware of and used in their profession. According to Meriam Webster (n.d.), a principle is a “com- prehensive and fundamental doctrine, or a fact of nature” (para. 1a). This word was chosen as a means of juxtaposing the practitioners’ domain exper- tise with published works like North Carolina State University’s Principles of Universal Design and Jakob Nielsen’s 10 Usability Heuristics for User Inter- face Design. (Mace, et al., 1997) (Nielsen, 1994) Unfortunately, this may have been a leading question— which, in turn, caused the SMEs to doubt the relevancy of their domain expertise, as described in Finding #1 of “UX Design for children”.
In order to A) address this potential flaw of the previous study and to B) return to the author’s initial research goal of creating guidelines for prac- titioners who are involved in the design of products for Deaf and hard of hearing children, the author designed a follow-up study which is described below: Seeking out the previous 13 practitioners in the space of user experience, accessibility, and education, the author re-interviewed these research partici- pants for 30-minutes using a very narrow set of probes which solely focused on the concept of human-centered design guidelines.
The interview questions existed on a scale of semantic probing which started with high-level questions about the participants’ knowledge of guide- lines and gradually worked towards domain-specific questions which sought to uncover the participants’ knowledge and opinions about the ultimate tar- get of the research study—e.g., Design guidelines for middle school-aged Deaf and hard of hearing students. The interview transcripts were then coded inductively using Values Coding in order to determine the values, attitudes, and beliefs of the participants (Saldaña, 2013, p. 111).
The author’s goal with this methodology was to better understand the types of information that practitioners in user experience, accessibility, and educa- tion might need to successfully design products for Deaf and hard of hearing students. Through this method, the author sought to create a reusable frame- work that will allow practitioners to examine the guidelines they use today, and to learn how to create the next generation of multidisciplinary-informed design guidelines.
The probes were as follows:
Q1. When you hear the word “guidelines”, what type of information would you expect to see?
Q2. If you knew that “guidelines” were created for designers, would you expect to see different information? If so, what type of information?
Q3. During the [previous] interview, we talked about empowering design- ers who create products for Deaf children. If you were reviewing “Deaf Design Guidelines”, would you expect to see different information? If so, what type of information?
Q4. If you were responsible for creating guidelines for designing Deaf middle school assessments, what type of process would you follow?
88 Polanco Jr and Liu
Research Participants In alignment with the previous study design of “UX Design for Chil- dren”, interview participants were categorized into two classifications: designers and subject matter experts. The hypothesis was that intervie- wees of the same classification would provide similar responses to each other.
It should be noted, however, that this delineation of interviewee categories is not a perfect one. For example, one interviewee self-identified as both an Engineer and a Designer. However, his described daily actions leaned more heavily towards development, not design—as such, he was classified as a subject matter expert.
Additionally, out of the 13 participants who were interviewed in the pre- vious study, only 12 were able to be interviewed for this study (Table 1). The participant not re-interviewed for this study was the Director of the Motion Light Lab. All interviewed participants chose to retain their previ- ously self-identified job titles for the purposes of this study, despite some of them changing job titles and employers between the previous study and this current study. For example, the Director of User Experience changed employ- ers and is now the Head of Accessibility within a new employer’s design organization, but still self-identified as the former.
Table 1. The distribution of the participants’ self-identified job titles. (Polanco Jr & Liu 2023).
Manager of Technical Content Design *Could be classified as designer or SME due to providing a hybrid job title.
RESEARCH RESULTS The responses to the research interview have been aggregated at the question- level and have similarly been delineated by classification type.
Q1. When You Hear the Word Guidelines, What Type of Information Do You Expect to See?
Amongst the designers, four out of six saw guidelines as being a list of requirements or things that should be done, usually to reach some level of compliance. Two of the six designers (the Lead R&D UX Researcher and the Director of Curriculum Development) saw guidelines as a type of referential material or starting point that could inform the design process.
The majority, however, saw guidelines as being a list of requirements or things to be done, usually to achieve some level of compliance. Due to the
Multidisciplinary Framework for Creating the Next-Generation 89
question being intentionally general, five out of the six designers did not men- tion a specific compliance entity or standards body was mentioned. The one remaining participant (the Senior Software Engineer / Accessibility Designer) quickly began to ground his response in the context of the Web Content Accessibility Guidelines (WCAG). In his opinion, WCAG appeared to be more of a list of things that should not be done, whereas he would prefer guidelines to be a list of things that should be done.
Unlike the designers, most of the subject matter experts felt that guide- lines were simply recommendations or steps to be successful; only one SME, the Sr. Software Architect, alluded to guidelines being used to satisfy an administrative person (i.e., compliance).
Q2. When You Hear That Design Guidelines Were Created for Designers, Would You Expect to See Different Information?
When asked to describe what kind of information they would expect from a set of design guidelines, the consensus across the designers and SMEs was “it depends.” The variance in expectations was similar, but slightly different between some of the respondents.
Amongst the designers, the UX Lead for Accessibility, the Director of User Experience, and the Senior Accessibility Manager alluded to the idea that dif- ferent disciplines of design may need different design guidelines. For example, it was posited that UX design guidelines may need information about func- tionality, but visual design guidelines may need to provide information about color contrast.
When the same question was asked of subject matter experts, all but one SME felt that more details should be provided for design guidelines than the previously nondescript / general guidelines. The specifics of these addi- tional details differed from SME to SME and were further different than those provided by the designers.
Examples include: the Research Scientist believed that the skill level of a designer may necessitate additional visual aids to explain more complex concepts. The Sr. Software Architect, alternatively, expected to see a lot of user stories. His thinking was that general guidelines that simply say what to do or not to do are insufficient, because they do not explain the reasons why these guidelines exist and who they impact. The Certified Teacher of the Deaf offered up a suggestion for additional information that no other practitioner offered: culture. Surprisingly, the Senior Software Engineer said that he expected no difference between design guidelines and general guide- lines; he was the only respondent who expected no difference between general guidelines and design guidelines.
Q3. During the Interview, We Talked About Empowering Designers Who Create Products for Deaf Children. If You Were Reviewing “Deaf Design Guidelines”, Would You Expect to See Different Information? If So, What Type of Information?
This third question gets closer to the author’s expectations of what practi- tioners might be interested in from a set of hypothetical set of DHH Design
90 Polanco Jr and Liu
Guidelines. Notably, both designers and SMEs made references to wanting to see information related to accessibility, with the UX Researcher making an explicit reference to WCAG.
Surprisingly, the Certified Teacher of the Deaf alluded to color contrast compliance as being necessary for a successful DHH student user experience. This was the first and only time that visual design was described as an imper- ative for DHH users. She described how something as seemingly innocuous as the color difference between a scene (i.e., virtual backgrounds in video calls, or colored walls in physical spaces) and a signer’s clothing could dras- tically inhibit a DHH student’s comprehension of signed language, such as ASL. She would go on to express a desire to see this type of multi-disciplinary, accessibility-compliance thinking in the creation of Deaf Design Guidelines.
While not directly contradictory, it was an interesting juxtaposition to hear the UX Lead for Accessibility be dismayed that visual disabilities often appear to be overrepresented in today’s accessibility guidelines and sought something different. In her estimation, Deaf Design Guidelines should be less concerned with how things look. It should be noted that of all the practitioners that were interviewed in this study, the UX Lead for Accessibility was the only interviewee who self-identified as being part of the DHH spectrum.
That said, the sentiment across both the designer and SME classification of participants was three-fold:
1) All but one SME expected different information from Deaf Design Guidelines than generic design guidelines. 2) They sought information that they could not intuit themselves without personally having a connection to the Deaf experience, and 3) There was a clear acknowledgement that the experience of being Deaf or Hard of Hearing is varied and should not be approached homogenously.
As far as the first point goes, the one SME who did not expect different information was the Special Education Teacher. In this case, they did not challenge that different information would be needed. Instead, they offered that it was outside the realm of their expectations and was emphatic that this was a question that DHH users should answer for themselves.
Regarding the second point, the Director of Curriculum development con- veyed a desire to see something that could simulate the DHH experience, akin to how color blindness simulators exist for exploring designs for visually impaired users.
On the topic of the third point, the Sr. Software Architect wanted these Deaf Design Guidelines to either: A) have complete coverage for the entire range of the Deaf and Hard of Hearing experience, or B) if this was not possible, have the gaps in coverage be explicitly stated as a limitation of the guidelines. To that end, he believed that these theoretical Deaf Design Guide- lines could serve as an exemplar for the creation and maintenance of future accessibility guidelines. This call for a gap analysis of the Deaf Design Guide- lines and other accessibility guidelines was similarly echoed by another SME, the Research Scientist, who referred to this analysis as a “transparency state- ment.” In their estimation, this would give greater credibility and validity
Multidisciplinary Framework for Creating the Next-Generation 91
to the Deaf Design Guidelines— as would use cases and exemplary designs. In these ways, both the Research Scientist and the Sr. Software Architect provided very similar responses.
Q4. If You Were Responsible for Creating Guidelines for the Design of Deaf Middle School Assessments, What Process Would You Follow?
Finally, with respect to this last question, every designer and SME stressed the importance of engaging experts who were knowledgeable of what it means to be Deaf or Hard of Hearing, either through first-hand experience or through being deeply embedded in the DHH community.
That said, while every practitioner alluded to engaging DHH experts, it is inconclusive as to whether every practitioner would work with DHH middle-school students directly. The interviewees were inconsistently probed to elaborate on what an expert meant to them. For examples:
The Lead R&D UX Researcher stated that he took it for granted that designers would be considered amongst the list of possible DHH experts.
The Director of User Experience and the Certified Teacher of the Deaf K-12 both stressed the importance of engaging with teachers who work with DHH students. The Director also claimed that every K-8 teacher she knew had to in essence become a designer out of necessity for meeting their students’ needs.
Only the Special Needs Teacher offered that she would personally work in a school for Deaf children so that she could directly ask DHH students what issues they face and what expectations they may have.
Only the Manager of Technical Content Design described working with experts in assessment delivery platforms. Their reasoning was to facili- tate stronger quality control through better understanding which assistive technologies are used within assessments.
The Senior Software Engineer/Accessibility Designer uniquely felt like DHH Middle school Assessment Design (DHH-MSAD) Guidelines would be best expressed as a WCAG guideline. As a result, a DHH expert in his opinion would consist of an individual who is responsible for W3C (World Wide Web Consortium) standards, e.g.. someone who is part of a W3C accessibility working group.
When asked to elaborate, the Sr. Software Architect emphatically stated that he probably would not engage a designer in the creation of these DHH-MSAD Guidelines. He would later amend his response to say that he would only engage designers who were close to the DHH experience, but his experience is that designers tend to operate more from online sources versus first-hand experience with the communities they design for.
CONCLUSION
The author still believes that a set of design guidelines are necessary to ensure that equitable and human-centered approaches are used in the design of prod- ucts for Deaf and Hard of Hearing users, especially in the design of middle school educational assessments. After interviewing 12 practitioners in the
92 Polanco Jr and Liu
space of user experience, accessibility, and education for a second time, the author’s belief in Deaf and hard of hearing Middle school Assessment Design (DHH-MSAD) guidelines remains steadfast. Unfortunately, the process by which these DHH-MSAD guidelines emerge still requires further investiga- tion; notably because the amount and types of experts that need to be engaged to create these guidelines was not conclusively identified in this latest round of interviews.
Fortunately, the interviews proved that the process of developing DHH- MSAD requires a multi-disciplinary effort across multiple domains of exper- tise. To the extent that this result is clear, the author’s research method of using a scale of semantic probing—which starts by eliciting general knowl- edge and works towards uncovering domain-specific outcomes—coupled with Values coding appeared to be successful.
To that end, this multidisciplinary framework may also prove to be com- patible with other researchers who seek to develop the next-generation of human-centered design guidelines for other user populations.
REFERENCES
Abdelazeem, B., Abbas, K. S., Amin, M. A., El-Shahat, N. A., Malik, B., Kalantary, A., & Eltobgy, M. (2022). The effectiveness of incentives for research participation: A systematic review and meta-analysis of randomized controlled trials. PloS one. doi: https://doi.org/10.1371/journal.pone.0267534 CDC. (2022). EHDI Annual Data. Retrieved from CDC.gov: https://www.cdc.gov/
ncbddd/hearingloss/ehdi-data.html ETS. (2020). How Tests and Test Questions are Developed. Retrieved Oct 2020, from ETS.org: https://origin-www.ets.org/understanding_testing/test_development Mace, R., Connell, B. R., Jones, M., Mueller, J., Mullick, A., Ostroff, E.,... Vanderhei-
den, G. (1997, April 1). THE PRINCIPLES OF UNIVERSAL DESIGN. Retrieved from The Center for Universal Design: https://design.ncsu.edu/research/center-fo r-universal-design/ Merriam-Webster. (n.d.). Principle Definition & Meaning. Retrieved 2023, from Merriam-Webster.com dictionary: https://www.merriam-webster.com/dictionary /principle NCES. (2020, May). The Condition of Education: Students With Disabilities.
Retrieved from National Center for Education Statistics: https://nces.ed.gov/pr ograms/coe/indicator_cgg.asp Nielsen, J. (1994, April 24). 10 Usability Heuristics for User Interface Design.
Retrieved from Nielsen Norman Group: https://www.nngroup.com/articles/ten-u sability-heuristics/ Pearson. (2018). Evaluating performance. Retrieved from Pearson VUE: https://home .pearsonvue.com/Documents/Security/pearson_vue_test_items_US.aspx Pizzo, L., & Chilvers, A. (2016). Assessment and d/Deaf and Hard of Hearing
Multilingual Learners: Considerations and Promising Practices. Retrieved from American Annals of the Deaf: https://www.jstor.org/stable/26235251 Polanco Jr, A., & Liu, T. (n.d.). UX Design for deaf children: How is it done now?
Should it change? Saldaña, J. (2013). Values Coding. In The coding manual for qualitative researchers (2 ed., pp. 110–115). SAGE.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human-Centered Design and User Experience
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-90-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional