Isabel L. Nunes, Duarte Patriarca, Ana Sofia Matos (2022). Usability-Based Mobile Phone Selection for Communications in Emergency Situations. In: Advances in Ergonomics In Design, Usability & Special Populations: Part II. AHFE Open Access, vol. 19. https://doi.org/10.54941/ahfe1001264
The work presented here describes the selection process of mobile phones with better usability for use in emergency situations in Portugal. In emergency management, communications are a fundamental asset for limiting the consequences of risky situations. Since mobile phones have revolutionized the way people communicate they can constitute an opportunity to be used in information exchange during emergency situations. In this study the mathematical methodology developed by (Jeelani, 2011) - which holistically represents human factors’ issues associated with the use of mobile phone in emergency - was adapted to the Portuguese context. This methodology allows the rating of mobile phones in terms of suitability for use in emergency situations, having the satisfaction of user needs as a main priority. The methodology used in this study comprehends three phases: identification of the more important features of mobile phones for use in emergency situations; determination of the selection factors relative importance, using Analytic Hierarchy Process; and usability testing of five mobile phones using Cognitive Walkthrough protocol, with 20 individuals.
Mobile phones are increasingly becoming part of our lives. These devices revolutionized the way people communicate and are still undergoing constant evolution offering many more functions beyond traditional voice calls. Text messages are currently a very common method of communication. On the other hand, the access to internet allows images and videos captured by mobile phones to be immediately shared worldwide. Social networks also became a privileged way of communication. Geographical location is often used to send geo-referenced information or to search for points of interest in the vicinity. Finally, most modern mobile phones allow the sending of position when an emergency call is made. Regarding these versatility of mobile phones, the fact that in emergency management, communications are a fundamental asset for limiting the consequences of risky situations and that cellular communications have not been officially implemented in the emergency management of many countries (Jeelani, 2011) it is desirable to study the adequacy of their use to support communication flow in emergency situations. One of the more important aspects is its usability in the context of emergency.
In fact, system usability is a characteristic that affects the effectiveness and success of any system, since it relates with the quality of the user-system interaction. Such interaction quality is particularly critical when systems are complex, and when the accuracy and timeliness of operation is decisive to the system outputs (Nunes and Simões- Marques, 2013). The usability of a system is dependent on the context of use, i.e., that the level of usability achieved depends on the specific circumstances in which the product is used. The context of use includes users, tasks, equipment and the physical and social environment, since all these factors directly influence the way how the Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4 interaction occurs (Simões-Marques and Nunes, 2012). Therefore, good usability is important because it is a characteristic of the product quality that leads to improved product acceptability, increased user satisfaction, improved product reliability and it is also financially beneficial to companies (Nunes, 2006).
In order to guide the selection of mobile phones for use during emergency management in The Bahamas (Jeelani, 2011) proposed a human-centered methodology. This methodology is based on the assessment of 10 criteria, that include a combination of mobile phones physical characteristics and usability attributes. The criteria are the following: Audio distinctness, Text entry method, Usability, Screen size, Portability, Accommodation to environmental lighting, Grip, Battery life & type, Mobile phone cost and Durability. The following formula calculates an Overall Score (X) for a given mobile phone, which indicates how appropriate this is for use in emergency situations:
Equation reproduced from the original PDF.
tm = rating of the mth sub-factor regarding Battery life & type (m = 1, …, 3); bn = relative weight of the nth sub-factor regarding Battery life & type (n = 1, …, 3);
po = rating of the oth sub-factor regarding Portability (o = 1, 2); cq = relative weight of the qth sub-factor regarding Portability (q = 1, 2);
ur = rating of the rth sub-factor regarding Usability (r = 1, …, 5); ds = relative weight of the sth sub-factor regarding Usability (s = 1, …, 5).
The goal of the study presented here was to adapt to the Portuguese context the human-centered methodology developed by (Jeelani, 2011). According to (Pordata, 2014), a Portuguese statistics data source, there are almost 20 million mobile phones in Portugal, for a population of 10 million citizens. The large number of these devices allows individuals to be no longer dependent on landline communications in situations of emergency, while offering an opportunity to use alternative means of reporting and characterizing incidents and their location. The complete study can be found in (Patriarca, 2013).
METHODOLOGY
The methodology used to select the most suitable mobile phone for use in emergency situations followed the approach proposed by (Jeelani, 2011). The scheme is depicted in Figure 1, and it consists of three phases.
Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
Figure 1. Methodology scheme to select the most suitable mobile phone for use in emergencies
In Phase 1 the most important characteristics of mobile phones for use in emergency situations were identified. These characteristics were highlighted during a focus group interview with emergency professionals from the following Portuguese Entities: Fire Departments, Civil Protection and National Guard. The individuals (N=35) also answered a questionnaire to collect information about: what mobile phones they used, what are the experience of using them during emergency situations and what improvements they would like to see implemented on mobile phones. The questionnaire had 19 questions, grouped into 5 categories: Personal data; Experience of communication with mobile phones; Assessment of mobile phone; Other communication devices; and Suggestions. A similar questionnaire was also applied to 155 Portuguese users of mobile phones.
In Phase 2 after the identification of the mobile phone selection factors and the definition of their relative weights were performed using Analytic Hierarchy Process (AHP). The pairwise comparison was carried out by the same individuals from the Entities referred before. Finally, the adaptation of the methodology for classification of each selection factor was performed. The selection factors were divided in physical characteristics, satisfaction metrics and performance metrics.
In Phase 3 five of the most used mobile phones were classified according to this methodology. The methodology incorporates a usability test, using the Cognitive Walkthrough (CW) protocol. Twenty individuals participated in this study. The Cognitive Walkthrough protocol (Wharton et al., 1994) is a usability inspection method whose objective is to identify usability problems, focusing on how easy it is for new users to accomplish pre designated tasks. The reactions and comments of the users are recorded as the walkthrough proceeds.
RESULTS AND DISCUSSION
Phase 1 The identification of the following mobile phones characteristics for use in emergency situations, resulted from the focus group interview (emergency professionals):
Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
Geolocation;
Good battery autonomy;
Good grip;
Robustness and water, dust and shock resistance;
Ergonomic design;
Adaptation to environment lighting conditions;
Low weight;
Adequate size;
Good visual distinctness;
Good audio distinctness.
Except for “Geolocation” all the other factors coincide with the factors identified in the (Jeelani, 2011) methodology. Therefore this factor was added to the model.
These individuals (N=35) answered also a questionnaire. Note that this was a convenience sample. It was composed by individuals belonging to Fire Departments (54%), National Guard (26%) and Civil Protection (20%). The majority of the individuals (74%) were 46 years old or less and more than half had over 10 years of experience working in Emergency. The main functionalities of the mobile phones used by the emergency professionals’ sample is shown in Figure 2. From these 89% of the mobile phones had camera, 60% had mobile internet and only 46% had GPS.
Voice Text Internet Camera GPS
Figure 2. Functionalities of mobile phones used by emergency professionals (N=35)
Figure 3 illustrates the usage of other communication devices by the emergency professionals’ sample. Radio is the preferred mean of communication (83%), followed by landline phones (69%) and computer-based applications (46%).
https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4 Ergonomics In Design, Usability & Special Populations II
Satellite Phone Landline phone Radio Computer Other
Figure 3. Other communication devices used by emergency professionals (N=35) In this phase a questionnaire was also applied to a sample of common mobile phones users (N=155). The questionnaire was answered by a convenience sample. 82% of the sample had an age under 36 years. Regarding functionalities all phones offered voice calls and text messaging, 92% had camera, 78% had mobile internet and only 48% had GPS. The distribution of other communication devices used by this population is depicted in Figure 4. Computer-based applications were the preferred mean to communicate (97%), followed by landline phones (52%).
Satellite Phone Landline phone Radio Computer Other
Figure 4. Other communication devices used by common users (N=155)
Phase 2 As referred before the human-centered methodology proposed by (Jeelani, 2011) is based on 10 criteria which, according to the author, were selected considering only features which were publicly available on end user hand- held communication devices at the time of his study. As suggested by (Jeelani, 2011) as mobile technology advances and as more innovative features are incorporated in the design of future devices, this list of device selection factors can change in order to better reflect the latest technological advances. Therefore, in consequence of the results of Phase 1, a new factor was added to the methodology – Geolocation. Figure 5 presents the selection factors which
https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4 Ergonomics In Design, Usability & Special Populations II were considered in the study.
Figure 5. Factors to select mobile phones. Adapted from (Jeelani, 2011)
In order to determine the weighted importance of each selection factor and sub-factor the Analytic Hierarchy Process was used. Five experts belonging to Fire Departments, Civil Protection and National Guard participated in the pairwise comparison. A double scale of 1-9, from “Extremely Strong” to “Equally Strong” was utilized for this purpose (see Figure 6). A rating of 1 (Equal) indicates that both selection factors (Factor A and Factor B) are equally important. A rating of 9 (Extreme) in the Factor B side indicates that Factor B is 9 times more important than Factor A.
Strong Very Strong Extreme
Equal 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Factor A Factor B Extreme Very Strong Moderate Strong Moderate
Figure 6. AHP pairwise comparison
Figure available in the original PDF paper.
After all opinions were collected Expert Choice® version 11 software was used to calculate the relative weight for each factor. The results are presented in Table 1. The “Audio distinctness” factor was considered the most important factor and the “Durability” the less important one, which is in agreement with (Jeelani, 2011) results. The relative weight of the other factors doesn’t match the results of (Jeelani, 2011).
Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
Table 1: AHP results for mobile phone selection factors Selection Factors Relative Ranking
The relative weights for the selection sub-factors are presented in Table 2. “Talk time”, “Volume” and “Ease to learn” are the most important sub-factors in their category.
Table 2: AHP results for mobile phone selection sub-factors Selection Factors Sub-Factors Relative Weight Ranking
According to the methodology, the factors and sub-factors were and grouped based on physical characteristics (green), satisfaction metrics (blue) and performance metrics (yellow) rated on a scale 1-3 (see Table 2). The criteria for rating each factor or sub-factor are also presented in the Table 3.
Table 3: Rating criteria for selection factors
Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
Phase 3 In this phase the 5 most used mobile phones (based on questionnaires conducted in Phase 1) were assessed. The mobile phones tested were two touch smartphones (A, D), one keyboard smartphone (B) and two basic cell phones (C, E).
Once all the data (physical characteristics, satisfaction metrics and performance metrics) were collected, it was converted to the scale 1-3, according to the criteria presented in Table 3. All the results of this Phase are presented in Table 4.
The physical characteristics of each mobile phone were assessed by the analyst based on the technical specifications provided by the respective manufacturer.
The satisfaction metrics (subjective measures) and the performance metrics (objective measures) were collected during a usability test using the Cognitive Walkthrough protocol performed on a sample of 20 individuals. The greater the number of participants the lower is the error and more accurate are the obtained results (Tullis and Albert, 2008). The test was performed using a convenience sample. The average age of participants was 27.8 years (standard deviation = 7.5 years). 85% of participants were male and 15% female. Only 35% of participants reported the use of mobile phone to communicate an emergency situation.
The usability tests were conducted in laboratory with proper lighting, sound and thermal conditions, providing a comfortable environment to participants. At the beginning of the session a version of the CW protocol was given to each participant, as well as a brief explanation of the procedure. During the explanation all doubts presented by Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4 participants were clarified, to minimize unwanted effects in test results. Before performing the tasks participants were allowed to familiarize themselves with the features of each mobile phone, such as making a voice call to assess the distinctness of the sound, send a text message and use the camera. Usability testing was conducted by a participant at a time, so that the analyst could properly evaluate the performance of each participant and gather all the necessary information. To reduce memorization bias and to obtain more accurate results the counterbalancing technique was used. This technique consists of altering the order of the tasks for each mobile phone (Tullis and Albert, 2008). Data collection was performed using the software Microsoft Excel ®, where all values of usability metrics evaluation were recorded. Tasks were performed with the mobile phones in flight mode in order to prevent false calls to the emergency number 112 (# emergency service).
The CW protocol comprised three tasks, which were selected based on the emergency services currently available and services that may become available in the near future. The tasks were the following: Task 1- Simulate a call to the emergency number 112. This task tested the currently available method of contact for
reporting emergencies.
Task 2- Take a photo and simulate sending it to the emergency number 112. This task tested a new service that could eventually become available in the near future, improving the information available to Emergency Management Entities, by combining a georeferenced emergency context picture.
Task 3- Write the following text message "Help I'm closed in the Ergonomics Laboratory of the Faculty of Science and Technology" and simulate sending it to the emergency number 112. This task tested a new method of contact in an emergency situation, which might become available in the near future, offering the advantage that this communication can be used when the cell network signal strength is weak or the mobile phone battery charge is low.
Regarding the results all participants were able to successfully perform all the planned tasks. Figure 7 presents the average time needed by the participants to complete each task. Note that the tasks are not exactly comparable since they don’t ensure the same type of outcome. For instance, the duration of task 1 is the lowest (average duration 2.2 s), however this task only assessed the time for dialing the emergency number and not the time to establish a voice call and to convey a message to the operator. The data gathered was used to assess the “Efficiency of use” sub- factor, which will be explained bellow.
Figure 7. Average time (and standard deviation) needed to complete each task Regarding the subjective measures the data were obtained using linguistic scales from “Very Bad” (or “Very Difficult”) to “Very Good” (or “Very Easy”) in the answer to questions, such as:
Figure available in the original PDF paper.
How do you rate the mobile phone sound distinctness?
Very Bad Very Good Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
1 2 3 4 5
Regarding the objective measures (“Efficiency of use” and “Error frequency”) the first step was to identify the best statistical distribution to be fitted to each task, considering a sample size of 100 (20 individuals x 5 mobile phones). Once obtained the best statistical distribution, the 33rd and 66th percentiles will allow us to quantify the border line of the scale 1 to 3. (Jeelani, 2011) used a Normal Distribution to fit to each task data. However, in this study, after visual analysis followed by Chi-Square Distribution fitting test to a Normal Distribution, the fitting was rejected with a p-value < 0.05, for the three tasks of each objective measure. On the left side of Figure 8 the frequency distribution for task 1 of “Efficiency of use” is shown along with the Normal Distribution line, where the lack of fit is notorious (Sturges rule was used to obtain the number of classes). A Gamma Distribution was then fitted to the same data and its cumulative distribution represented on the right side of Figure 8 (Chi-Square Distribution fitting test with a p-value of 0.423). The 33rd and 66th percentiles represented by p1 and p2 respectively, allowed to achieve the scale 1 to 3, being the corresponding values in seconds: (2.50 – 8.00), (1.65 – 2.50) and (0.00 – 1.65).
Figure 8. Frequency distribution with Normal Distribution fitting line (left) and Gamma cumulative distribution (right), for task 1
Figure available in the original PDF paper.
This procedure was applied to the remaining tasks of both performance metrics. With the new scale results an average for each task per mobile phone was calculated. From the average of the three tasks’ results per mobile phone we derived the ratings for “Efficiency of use” and “Error frequency” shown in Table 4.
As referred before, Table 4 shows the assessment for each selection factor for the five mobile phones tested as well as the Overall Score provided by the adaptation of Jeelani’s (2011) human-centered methodology to the Portuguese context. Smartphone 1 reached the highest Overall Score, followed very closely by Smartphone 2. Regarding the 4 most important selection factors, the assessment was:
- “Audio distinctness” – all 5 devices obtained identical ratings in this factor; - “Text entry method” – the 2 touch smartphones were the ones better rated regarding this characteristic; - “Geolocation” – only the 3 smartphones offered this feature; - “Usability” – the 2 touch smartphones were the ones better rated considering this factor.
Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
Table 4: Ratings and overall score for the five mobile phones tested Mobile phones
CONCLUSIONS
The study presented an adaption to the Portuguese context of the mathematical methodology developed by (Jeelani, 2011) which holistically represents human factors issues associated with the use of mobile phones in emergency. As happened in the study performed in The Bahamas this human-centered methodology for the assessment of mobile phone allows the rating of equipment in terms of suitability for use in reporting emergency situations, having the satisfaction of user needs as a main priority. This methodology is truly human-centered since all the phases involved user input, namely in the identification of selection factors and in the incorporation of usability (subjective opinions and operator-use objective measures).
Based on the results from interviews (Phase 1) a new selection factor (“Geolocation”) was added to the existing model. Therefore, there was the need to reevaluate the weighted importance of each selection factor using AHP. The AHP results indicate that "Audio distinctness" was considered by respondents as the most important factor, since voice call is the most used communication method in the report of emergency situations. Regarding the "Battery life & type" factor respondents considered "Talk time" as the most important sub-factor. Both classifications are in agreement with the results obtained by (Jeelani, 2011). Regarding "Usability" respondents rated "Ease of learn" as the most important the sub-factors, probably because mobile phones should be easy to use so that users can start work with them quickly.
Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
In this study, based on the individuals’ population engaged, on the proposed tasks and on the set of devices analyzed, it was found that Smartphone 1 reached the best Overall Score, being the device with better characteristics, followed by Smartphone 2 (with a very close overall score). Although this study introduced a new factor in the assessment of mobile phone, the “Geolocation” and some rating criteria were adapted to the Portuguese context, the results are similar to those obtained by (Jeelani, 2011). It should be noted that this study was performed mainly to test the proposed methodology in another country, by adapting it to a different context. On the other hand, considering that the tests performed in the different phases of the methodology were based on a convenience sample the results don’t have statistical significance; therefore they cannot be extrapolated to the entire Portuguese population. Despite this it was possible to conclude that the human- centered mobile phone selection model proposed by (Jeelani, 2011) can be adapted to other contexts, which was one of the suggestions made by Jeelani in his work.
Acknowledgement: This work was funded by Fundação para a Ciência e a Tecnologia under the scope of project PEst-OE/EME/UI0667/2011. The authors want to thank Dr. Pamela McCauley-Bush (UCF, FL) for the challenge of applying in Portugal the methodology developed by Jeelani, and for the fruitful discussions held in the beginning of the project.
REFERENCE Jeelani, M. I. (2011) “The development of a human centered methodology for the identification of communication needs and the
assessment of hand-held communication devices used to support communication flow in high school consequence emergency management” Master of Science Thesis in Industrial Engineering. Florida: College of Engineering and Computer Science University of Central Florida Nunes, I.L. (2006) “Ergonomics & Usability - key factors in knowledge society”, ENTERPRISE AND WORK INNOVATION STUDIES, Volume 2, pp. 87-94, ISSN 1646-1223 Nunes, I.L., Simões-Marques, M. (2013) “SINGRAR Usability Study”, in Design, User Experience, and Usability. Design
Philosophy, Methods, and Tools (Second International Conference, DUXU 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part I), A. Marcus (Ed.): DUXU/HCII 2013, Jul 21-26, Las Vegas, USA, Lecture Notes in Computer Science (LNCS), Springer-Verlag Berlin Heidelberg, Part I, pp. 359-368 (ISI) Series ISSN 0302-9743, doi 10.1007/978-3-642-39229-0_66 Print ISBN 978-3-642-39228-3 Patriarca, D. (2013) “Usabilidade de dispositivos portáteis utilizados nas comunicações na gestão de emergência” Master of Science Thesis in Industrial and Management Engineering. Faculdade de Ciencias e Tecnologia- Universidade Nova de Lisboa (available at http://run.unl.pt/handle/10362/11391) Pordata (2014) “Assinantes / equipamentos de utilizadores do serviço móvel”. Available at http://www.pordata.pt/Portugal/
Ambiente+de+Consulta/Tabela Simões-Marques, M., Nunes, I.L. (2012) “Usability of Interfaces”, in: Ergonomics – A Systems Approach, Nunes, Isabel L. (Ed.)
InTech. pp.155-170 Tullis, T., Albert, B. (2008) “Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics” Morgan Kaufmann Publishers, Burlington, EUA Wharton, C., Rieman, J., Lewis,C., Polson, P. (1994). “The cognitive walkthrough method: a practitioner's guide”, in: Usability
Inspection Methods, J. Nielsen and R. L. Mack (Ed), pp. 105-140, John Wiley & Sons, ISBN 0-471-01877-5
Ergonomics In Design, Usability & Special Populations II https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2107-4
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Ergonomics In Design, Usability & Special Populations: Part II
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2107-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional