Tania Calle-Jimenez, Bryan Munoz (2023). Deep Analysis of the Web Accessibility and Usability of Videoconferencing Platforms for Blind People. In: Usability and User Experience. AHFE Open Access, vol. 110, pp. 126–132. https://doi.org/10.54941/ahfe1003184
During the COVID 19 pandemic, the use of virtual videoconferencing platforms increased considerably. Most of the group activities associated with this are conducted remotely, whichinvolves technological change affecting people, especially blind people. These videoconferencing platforms are hosted on the web, causing visually impaired users to enter a new era of communication with all its accessibility barriers and usability issues. For this reason, a systematic literature review (SLR) was conducted which allowed us to determine the level of web accessibility that these platforms have, the studies and all the implementations that are being conducted to mitigate this problem. In addition, the web accessibility of the three most widely-used videoconferencing platforms for both work and education was evaluated. Automated tools were used, individual tests were performed, accessibility guidelines (WCAG 3.0) were applied to determine the level of web accessibility, and usability tests were performed on a group of visually impaired individuals. The methods used were: Think Aloud (TA), the System Usability Scale Test (SUS) and Nielsen’s 10 Heuristic Usability Principles. The results of these tests and methods indicate that the selected videoconferencing platforms do not meet the minimum requirements for web accessibility, and that the level of usability is low. Therefore, changes need to be made and new implementations need to be developed with regard to these tools, so that visually impaired users can navigate the platforms smoothly and without any problems.
In December 2019, the world was informed that there was a virus causing acute respiratory syndrome, thus generating the Covid-19 pandemic. The virus affected all types of social interactions such as those found at work, in education, and in sports (Gallardo, 2020). This caused an urgent and unforeseen transformation, whereby homes were transformed into offices, classrooms and university classrooms, the main means of communication being virtual videoconferencing platforms (Gallardo, 2020). During this time, some companies and educational institutions offered alternatives and a variety of technological resources (computers, laptops, and the Internet) to sectors, groups and individuals interested in continuing to work, study or perform other activities to prevent the spread of this virus (Ramayo, 2019).
Deep Analysis of the Web Accessibility and Usability 127
According to the statistics, there were approximately 1.3 billion people in the world who were engaged in labor activities or educational processes. Consequently, due to this technological change, there were several barri- ers to be overcome in order for individuals to comply with the activities that were entrusted to them virtually, especially for people with disabili- ties (Bonilla-Guachamin, 2020). The people involved had to use web-hosted applications to communicate. In this regard, web accessibility and usabi- lity played a significant role since most digital platforms are not sufficiently intuitive and accessible (Ramiro, 2020). Currently, companies such as those employing independent development teams put web accessibility on the back burner because it is too costly to invest in training or to acquire the technolo- gies that make a site more accessible and functional.Therefore, in this study it was decided to evaluate the web accessibility of the most commonly-used videoconferencing platforms: ZOOM, Webex, and MS-Teams (Aulla, 2020), with the help of the automatic accessibility evaluation tools TAW, WAVE and ARC Toolkit. Additionally, individual tests were performed and the WCAG 2.2 and 3.0 (W3C, 2021) guidelines were applied to complement the results of the automatic tests and thus better understand the level of web accessibi- lity. In terms of the usability tests, three methods were applied: Think Aloud (TA), which involves performing a set of tasks while the users who execute them think aloud about their feelings; the System Usability Scale (SUS) que- stionnaire, which is used to evaluate the usability of a system by means of ten questions (Broke, 2020); and finally, usability evaluation with heuristics was performed with the help of Nielsen’s 10 Heuristic Principles (Nielsen, 1994).
RELATED WORK
In the research “Evaluation of accessibility in teleconferencing systems for low vision users during COVID19” (Acosta-Vargas, et al. 2020), an acces- sibility evaluation of the ten most used videoconferencing platforms during the COVID-19 pandemic was carried out, based on the needs of people with low vision and aligned to the WCAG 2.1 and 2.2 accessibility guidelines. Additionally, it is mentioned that there is no specific tool to evaluate the web accessibility of this type of platform. Consequently, seven phases were applied to perform a manual evaluation: (1) Select videoconferencing platforms, (2) Explore these platforms, (3) Select evaluation scenario, (4) Select group of users, (5) List accessibility barriers for this group of users, (6) Record and analyze the results, and (8) Suggest improvements. As a result of this analysis, it became evident that none of the videoconferencing platforms considered is accessible because it does not meet the minimum accessibility requirements. The best rating is given to ZOOM, followed by MS-Teams and Hangouts (Acosta-Vargas, et al. 2020). On the other hand, in the research “Method for Assessing Accessibility in Videoconference Systems” (Aulla, 2020) the auth- ors indicate that, in recent years, the use of videoconferencing platforms has increased due to the Covid-19 pandemic. During this time, all social activities have been carried out virtually, affecting people with some type of disability because most of these systems are not accessible to such individuals. For this reason, in this research an evaluation of the manual accessibility of the six
128 Calle-Jimenez and Munoz
videoconferencing platforms most used by users in 2020 was carried out by applying the WCAG 2.1 and 2.2 guidelines. The result was that the most accessible videoconferencing platforms are: 1) ZOOM, 2) MS-Teams and 3) Google Meet. In the research “Accessibility Evaluation of Video Confe- rencing Tools to Support Disabled People in Distance Teaching, Meetings and other Activities” (Hersh, et al., 2020), the authors state that there are many accessibility and usability problems associated with videoconferencing platforms for users with disabilities. These barriers in the educational envi- ronment cause many pedagogical difficulties to arise because technology supports teaching and learning in an essential way. Previously, users did not realize the potential of these platforms, and it was only with the onset of the Covid-19 pandemic that more such systems began to be developed. This study indicates that none of these videoconferencing platforms is accessible, and that there are no comprehensive studies on the degree of accessibility and usability of these tools. However, the authors present three recommendations to fill this gap: 1) Search for tools that help to automatically evaluate accessi- bility and usability; 2) Usability testing; 3) Educate developers on accessibility issues. With regard to implementation issues, there is a need to provide 1) Subtitles in the native language of each user and 2) SignWriting (Hersh, et al., 2020). Next, in the research “Accessible Video Calling: Enabling Nonvisual Perception of Visual Conversation Cues” (Shi, et al., 2019) the authors cre- ate a prototype of non-visually accessible video calls (NAVC). This is a tool that detects visual and audio cues (movie soundtracks) through AI to express attention, agreement, disagreement, happiness, surprise among other emoti- ons among visually impaired users. However, a very satisfactory result was not obtained, in that people with visual impairment managed to identify some signals such as attention and agreement, but had difficulty when it came to identifying negative signals. This worried the researchers as they did not want false positives to occur in a conversation and for misunderstandings to occur. Ultimately, this research shows a new way to realize accessible video confe- rencing platforms with the use of AI (Shi, et al., 2019). Finally, the research “Videoconferencing applications: accessibility study” (Ramiro, 2020) indi- cates that not all videoconferencing platforms are accessible because they do not have the necessary and basic options that can be used by people with visual and hearing disabilities. Visually impaired users present several diffi- culties that can be partially mitigated with the implementation of keyboard shortcuts, audio descriptions, screen readers (both internal and external) and with the auditory content presented in their native language. Additionally, it evidences four unique principles to ensure that videoconferencing plat- forms are accessible: 1) Allowing the user to adjust the systems. 2) Equivalent audio and video content. 3) Technical aids and full access via keyboard shor- tcuts. 4) Providing feedback and guidance within the system. The author is convinced that little by little this type of platforms will evolve in terms of accessibility issues because the Covid-19 market has expanded, competi- tion has increased, and therefore this will lead to continuous improvement (Ramiro, 2020).
Deep Analysis of the Web Accessibility and Usability 129
METODOLOGY The accessibility was evaluated with the automatic and individual accessibi- lity of the videoconferencing platforms: ZOOM, Webex, and MS Teams, to determine their current level of accessibility. Automatic accessibility evalu- ation tools were used to automatically evaluate the web accessibility in the videoconferencing platforms: WAVE, TAW, and ARC Toolkit. These tools deliver in a matter of minutes a complete and detailed report of all errors, warnings, and problems to be checked individually. Additionally, individual evaluation of accessibility to perform this evaluation, apart from the items to be checked, a list of the most common web accessibility barriers that a visu- ally impaired person may encounter when using or manipulating this type of system was prepared, associated with the guidelines, principles, and success criteria of the WCAG 2.2 (W3C, 2021).
It should be emphasized that WCAG 3.0 does not precede WCAG 2.2. This is a new version with different structure, characteristics and qualification methods [8]. In this sense, the levels of conformity of each one are incor- porated with the accessibility barriers mentioned above. Thus, it is possible to create a list with which to work to determine the level of accessibility of videoconferencing platforms with respect to the WCAG 3.0 guidelines (W3C, 2021). Regarding to perform the usability tests, the TA method was used. This resulted in the development of a list of tasks that each user must perform with regard to each videoconferencing platform considered.
Additionally, the SUS questionnaire was applied. This consisted of ten que- stions, which are evaluated by means of a Likert scale involving five points: (1) Strongly disagree, (2) Disagree, (3) Neutral, (4) Agree, and (5) Strongly agree. If the result of this survey is equal to or higher than sixty-eight, the system is considered acceptable in terms of usability. However, if the score is lower than this, it indicates that there are several usability problems that should be dealt with (Broke, 2020).
Finally, the heuristic usability evaluation was used to perform this evalu- ation, use was made of Jakob Nielsen’s 10 Heuristic Principles of Usability, which aims to maintain an interactive interface and avoid a high percentage of usability problems (Nielsen, 1994).
RESULTS AND DISCUSSION
In summary, the following results were found for each videoconferencing platform with respect to its accessibility, usability, WCAG 3.0 Guidelines and possible solutions, see Table 1.
Web accessibility is very important for people with visual disabilities. Statistics indicate that a large percentage of people with disabilities go through a labour or educational process and that due to the barriers that exist when using web systems, most users with disabilities must abandon these processes. Consequently, it is important that companies invest in trai- ning and technological tools to increase or develop inclusive and intuitive systems for any type of user. Most of the scientific articles taken as a reference
130 Calle-Jimenez and Munoz
Table 1. Problems and solution on videoconferencing platforms.
Table available in the original PDF paper.
Platform Problem Solution Zoom Hierarchical structure should be improved.
Apply section 3.5. WCAG 3.0 (W3C, 2021)
Apply section 3.4. WCAG 3.0 (W3C, 2021) Zoom Contrast between text and background should be improved.
Zoom Text messages are not saved after the end of a meeting.
Apply section 3.4 WCAG 3.0 (W3C, 2021)
Webex Hierarchical structure is not correctly Designed
Apply section 3.4 WCAG 3.0 (W3C, 2021).
Webex Contrast between text and background colour should be improved.
Webex Interface should be user-friendly Apply 8th Nielsen Usability Heuristic Principle (Broke, 2020). Webex Content should be understandable to the user.
Apply section 3.2 WCAG 3.0. WCAG 3.0 (W3C, 2021) & 8th Nielsen Usability Heuristic Principle (Broke, 2020). Webex Font size in footer should be increased.
Apply 4th Nielsen Usability Heuristic Principle (Broke, 2020). Webex Automatic subtitles only play in English.
Apply section 3.2. WCAG 3.0 (W3C, 2021).
Apply section 3.3 WCAG 3.0 (W3C, 2021). MSTeams Content must be understandable and fully visible to the user.
MSTeams Error messages that are displayed should be improved.
Apply section 3.6 WCAG 3.0 (W3C, 2021) & 6th Nielsen Usability Heuristic Principle (Broke, 2020). MSTeams This platform does not present subtitles.
Apply section 3.3 WCAG 3.0. (W3C, 2021).
for this work are based on WCAG 2.1 and 2.2, which are the most consoli- dated guidelines presently available. However, the decision has been made to include the WCAG 3.0 guidelines in this work because its structure makes it easier to understand and for its results to be interpreted. Apart from perfor- ming automatic and individual web accessibility tests, it considers usability and heuristics to speculate on the level of conformity beyond that found when evaluating the accessibility of a system. Based on this, we can say that, accor- ding to the data obtained and the analysis of the automatic, individual and usability tests, the videoconferencing platforms considered in this research are not accessible for people with visual impairment.
The evaluated platforms have 30% accessibility errors, 50% contrast errors and 20% warnings and accessibility problems that must be verified manually. On the other hand, when performing the individual tests with the help of the WCAG 2.2 and 3.0 guidelines and the level of conformity, we found that ZOOM and MS Teams are the most accessible platforms in that
Deep Analysis of the Web Accessibility and Usability 131
they pass twenty of the twenty-six selected web accessibility barriers for peo- ple with visual impairment. It should be emphasized that a difference was found between these two platforms in that ZOOM has implemented subti- tles that are automatically reproduced in the native language of each user, which is a positive development. In this research work usability is important. Although accessibility and usability are different terms, they have a common goal: to make the user able to access information without any difficulty, and thus improve their experience when navigating within any web system. In this research, we collected all the results with regard to usability evaluations.
When analyzing them we found that ZOOM and Webex do not get a good rating in terms of usability issues, in that they have several problems such as: (1) their screen readers emit erroneous information and at certain points get stuck, (2) their web pages are not well structured, (3) there are no error messages, and (4) their design is not simple or aesthetic. On the other hand, the MS Teams videoconferencing platform has a higher rating, because it hosts its desktop application on the web, which causes the user to feel a sense of comfort, fluidity and confidence when performing any task within it. It is completely structured, its buttons have graphics that perfectly reflect its functionality, while the screen reader runs smoothly and presents error messages.
CONCLUSION AND FUTURE WORK The use of videoconferencing platforms has increased in recent years to prevent the spread of the Covid-19 virus. These platforms have been of significant help to users in employment and education. For this reason, it is important that these sites are accessible to all users, including those with disa- bilities, since most group activities are now carried out virtually. During the last few years, it has become evident that videoconferencing platforms are not accessible. However, web accessibility is a topic of little importance for entre- preneurs and development teams, since implementing accessibility within a system, platform or web tool is extremely costly and takes a long time to implement. Money must be invested to acquire the appropriate technology and above all to train developers who are the key feature in the planning, design, and implementation of web tools. On the other hand, based on this work, it can be concluded that none of the videoconferencing platform consi- dered is fully accessible to visually impaired people. These users need the help of assistive technologies such as screen readers, virtual narrators, and Braille keyboards, in order to access information and maintain a constant ability to navigate without any barriers to their progress. In addition, users with a disability level less than 70% indicate that they do not feel comfortable using these platforms because the interfaces are not intuitive or user-friendly, the keyboard shortcuts are not correctly defined, there are contrast errors, and the text is sometimes ridiculously small.
Although in this work, several accessibility evaluations were performed and different usability methods were applied, there is still a large field of study and tests that can be used to determine the degree of accessibility of a web system. In this sense, the present work can serve as a reference for future
132 Calle-Jimenez and Munoz
studies involving web accessibility, WCAG 3.0, usability and heuristic meth- ods. In addition, it can be directed to users with other types of disabilities, and to evaluate other types of platforms.
REFERENCES
Acosta-Vargas, P., et al. (2020) “Evaluation of accessibility in teleconferencing systems for low vision users during COVID-19,” RISTI - Revista Iberica de Sistemas e Tecnologias de Informacao, Volume No. 37, pp 205–218. Aulla, M. (2020) “Use of video conference tools for current work,” M. S.
thesis, Higher Polytechnic School of the Litoral, Guayaquil, Ecuador. Bonilla- Guachamín, J. (2020). “The two faces of education in COVID-19,” CienciA- mérica, Volume 9, No. 2, p. 89. Brooke, J. (2020). “SUS: A ‘Quick and Dirty’ Usability Scale,” Usability Eval. pp. 207–212. Gallardo, F. (2020). “The challenge of linking with the environment in times of pandemic covid-19,” Cienc. y Enferm., Volume No. 26, pp. 1–3. Hersh, M., Leporini, B. and Buzzi, M. (2020) “Accessibility evaluation of video
conferencing tools to support disabled people in distance teaching, meetings and other activities,” ICCHP Open Access Compendium “Future Perspectives of AT, eAccessibility and eInclusion”, pp. 133–139. Nielsen, T. (1994). “Ten Usability Heuristics”. Ramayo, F. (2019). “ICT in education: Implementation of videoconferencing in the Moodle platform of the Faculty of Technology and Applied Sciences of the National University of Catamarca”, pp. 106. Ramiro, J. (2020). “Video conferencing applications: accessibility study,” ITSMF
España, p. 22. Shi, L., et al. (2019). “Accessible Video Calling”, Proceedings of the ACM on Human- Computer Interaction, Volume No. 3, pp. 1–22. Testing de Software and QA – Qalified. (2022). Guide to accessibility tests. Webpage:
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-86-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional