Hung-Chun Chen, Sheue-Ling Hwang (2020). A Modified Failure Mode and Effects Analysis for In-Vehicle Speech Interaction Systems. In: Advances in The Ergonomics in Manufacturing: Managing the Enterprise of the Future. AHFE Open Access, vol. 12. https://doi.org/10.54941/ahfe100452
The purpose of this study was to identify the potential problems of an in-vehicle speech interaction system named Talking Car by using the techniques of risk assessment. In this study, there were 16 participants asked to complete the assigned tasks by controlling the Talking Car system while driving. By conducting the driving simulation experiment, the failure modes, the incorrect operations, the failure causes and the effects were observed and collected. Then, a modified Failure Mode and Effects Analysis (FMEA) method which took human operation into account was used to determine and analyze these failure modes and incorrect operations. After completing the analysis, the Risk Priority Number (RPN) is computed to detect which failure made a significant effect on driving safety. As a result, RPN and subjective questionnaire indicated that “System Sentences with Elicitation”, “Control Button on Steering Wheel” and “Partial System Interface” should be improved urgently to ensure driving safety.
Recently, with advances in technology and information acceleration, many drivers tend to use the systems of vehicle telematics to get information while they are driving. Telematics systems provide the drivers some useful functions when they are driving, such as GPS, phone and radio. However, using the telematics systems becomes a risky factor due to complex operating mode which may lead to driver distraction (Strickland, 2012). Among these distracting tasks, visual and manual tasks, such as selecting a phone number or adjusting the radio channel, seem to significantly raise the risk levels (Fitch et al., 2013). Since the development of the telematics systems in Taiwan is not mature enough, the design of current in-vehicle speech systems on market may have some disadvantages to cause a certain degree of distraction and errors. There are many previous researches about driver distraction and driving safety; however, very few researches investigated the errors as controlling the in-vehicle speech systems during driving. Failure Mode and Effects Analysis (FMEA) is a risk assessment tool to anticipate and mitigate possible failures, problems and errors in system, design, process and service (Lin et al., 2014). However, FMEA seldom analyzes from the viewpoint of human and related human errors (Yu et al., 1999). In addition to the failures of single component itself, human error also makes effects on the system. Hence, for operation safety, human operation should be included in FMEA. The purpose of this study was to identify the potential problems of the Talking Car system by using the appropriate techniques of risk assessment. The results of the analysis are available Ergonomics in Manufacturing (2020) for system improvement; drivers therefore can use the improved in-vehicle speech system more easily and pay more attention on driving.
BACKGROUND
Driving Distraction and In-Vehicle Speech Interaction System It is thought that driving is an activity which needs high attention resource to perform; drivers should put their hands on the steering wheels and eyes on the road. However, there are many secondary tasks make the drivers’ hands and eyes away from their original location, such as operating the in-vehicle systems that is a common way to cause driving distraction. According to National Highway Traffic Safety Administration (2010), distraction has defined as “a specific type of inattention that occurs when drivers divert their attention away from the driving task to focus on another activity instead”. NHTSA (2010) also elaborated that voice interface is one of the improvements and speech technology would be introduced to in-vehicle systems.
Although speech interface is regarded as a good choice for solving driving distraction, some researches indicated that drivers often drive worse than just driving when operating a speech interaction system (Barón and Green, 2006). Lee et al. (2001) showed that in-vehicle speech-based interaction system increases the cognitive load on the driver. There are also some issues about in-vehicle speech-based interaction system. Multimodal systems which combine speech modality with visual or tactile modality have been more common (Chen et al., 2010), and adding a visual display screen is one of the way to present multimodal interaction; however, it violates the need of keeping eyes on the road as driving. Unnatural and complex voice command is another problem of speech-based interaction system that makes users feel inflexible and hard-to-use (Hua and Ng, 2010).
Failure Mode and Effects Analysis There are many techniques of risk assessment, and failure mode and effects analysis (FMEA) is one of the most popular techniques. FMEA is a systematic method which is considered as a useful and powerful technique in assessing potential failures and preventing them from occurring (Shakar and Prabhu, 2001).The beginning of FMEA was first proposed at Grumman Aircraft Corporation in the 1950s (Sellappan and Palanikumar, 2013). According to Liu et al. (2013), the results of FMEA can help the analysts tofind, prioritize and correct the failure modes which have deleterious effects and improve its performance during the stage of design and production. Therefore, FMEA has been broadly used in various areas and different kinds of industries (Rhee and Ishii, 2003).
The risk priority number (RPN) method can be added into FMEA to evaluate the cause criticality of a possible defect and prioritize the failure modes for improvement (Puente et al. 2002). RPN is the multiplication of three indexes: severity index (S), occurrence index (O) and detection index (D) (Teng and Ho, 1996). RPN = S×O×D, where S refers to the severity level of the consequent effect, O refers to the probability of the cause occurred and lead to the failure mode and D refers to the likelihood of detecting a failure before it occurred actually. For each index, a 10-point scale is used to evaluate the risk factor and to obtain a RPN of every failure mode (SAE J1739, 2000). The RPN method helps to focus resources on high “risk” failure modes (Kmenta and Ishii, 2000). With respect to the score of RPNs, the failure modes can be ranked and then proper actions will be preferentially taken on the failure modes with higher RPNs to mitigate the risk.
METHOD
To know what kinds of failures and problems occurred when drivers use Talking Car while driving, the driving simulation experiment was conducted. Some information like failure modes, incorrect operation, failure causes and effects were collectedby experiment and used in a modified FMEA method which took human operation into account. Figure 1 shows the process of the method in this research.
Figure 1. Process of research method Experiment: Driving Simulation Experiment
In the experiment, there were sixteen participants, including ten males and six females, between 20-25 years old. All participants had car driving licenses but had no experience for in-vehicle speech interaction system. The entire experiment lasted almost one hour.
To identify the disadvantages as many as possible, all the functions of Talking Car, including music, radio, SMS and phone were used to design the experimental tasks. Subjects should complete all the tasks step by step. When the experiment was carrying on, a camera was set to capture the subject’s operations. There was something need to be recorded, for example, the reaction time of the brakes when the screen appeared a “STOP” message suddenly, the number of departures, vehicle speed, the number of incorrect operations, such as uttering the command without using interrupt control buttons, using incorrect control buttons and commands, etc. Besides, the total task completion time and the number of task failure were also recorded. After the experiment, subjects required to take an interview and answer the subjective questionnaire to give their opinions about why they made mistakes and the disadvantages of Talking Car.
A Modified FMEA Procedure for Operation Safety The disadvantages of the Talking Car system might influence the operation performance and driving safety when the drivers controlled the system. However, it was not comprehensive enough to explore these disadvantages only from the perspective of the system. Therefore, human operations need to be included in the risk assessment. And the modified FMEA procedure for the task of controlling the Talking Car system while driving is interpreted as following sections.
In the experiment, subjects need to control the Talking Car system while driving. Thus, all the human operations could be divided into two parts. One was about driving, named “Driving Task”, and the other was about controlling the Talking Car system, named “Controlling the System Task”. For “Driving Task” and “Controlling the System Task”, the failure modes were unsafe driving situations and incorrect operations that resulted in failing in controlling the system, respectively.
After determining the human operations, the analysts identified failure modes and incorrect operations by the videos and the experience. Since each failure modes and incorrect operations might have some causes, it is essential to determine which one was the major cause to lead to the failures. To collect such information, the analysts watched the videos and also explored the reasons from the interview which was taken after the experiments. The effects of the failure modes and the incorrect operations were listed as well. And then, scoring S, O and D should be allocated for each failure modes and incorrect operations. It should be noted that occurrence rate was not merely based on subjective judgment but on the probability obtained through the videos and the interview. By multiplying the severity, occurrence and detection rate, Risk Priority Number (RPN) of each failure mode and incorrect operation were calculated.
RESULT
The Result of Experimental Tasks In the driving simulation experiment, subjects need to complete assigned tasks by controlling the Talking Car system while driving. For these assigned tasks, the total task completion time was 18 minutes and 48 seconds (1128 seconds) in average. In addition, the failure rate of each task was calculated. In this study, the criterion of task failure was that the subject could not complete the tasks in his/her first attempt. The task of the highest failure rate is “Cancel the recognition” (81.25%), and the next is “Switch to radio function from SMS function” (62.5%). The problems which led to such task failures might be dealt with in the future.
The Result of the Modified FMEA for Operation Safety As mentioned before, the higher RPN is, the more urgent the improvement of the failure mode is. In this section, the disadvantages which need to be improved more urgently were explored respectively for the “Driving Task” and “Controlling the System Task” through the modified FMEA. For the “Driving Task”, the top three of RPNs are calculated and discussed. Further, the most important factors of controlling the system which led to “Extension of Reaction Time” and “Departures” are identified. It indicated that “System Sentences with Elicitation”, “Control Button on Steering Wheel”, and “Partial System Interface” which made a great effect on “Driving Task” need to be improved.
There were some differences between “Controlling the System Task” and “Driving Task”. The RPNs for incorrect operations in “Controlling the System Task” were lower than that for failure modes in “Driving Task”. Therefore, the highest RPNs for incorrect operations and their causes are considered. Accordingly, “Control Button on Steering Wheel”, “System Sentences with Elicitation and “The Timing of Uttering the Commands” were the parts of system which need to be improved.
Subjective Questionnaire By the questionnaire, requested the subject provided their opinions of disadvantages and suggestions for the Talking Car system. Table 1 shows the result of subjects’ opinions. The suggestions by the subjects were divided into five categories. According to Table 1, “System Sentences with Elicitation”, “System Interface” and “Functions of Control Buttons” are major parts of the subjects’ suggestions.
Table 1: Category of suggestions from subjective questionnaire Category of Suggestion Frequenc
Table available in the original PDF paper.
%
y System sentences with elicitation need to be improved. 36 30.8%
System interface is not intuitive enough. 25 21.4%
Functions of control buttons are not complete. 21 17.9% Basic operation of the system is defective. 19 16.2%
Layout of the control buttons on steering wheel is not good. 16 13.7%
CONCLUSIONS
This study is aimed to identify the disadvantages of the Talking Car system which need to be improved by using the modified FMEA method. Regarding the experimental results and analysis, there are indeed some significant disadvantages need to be reduced in current Talking Car system, namely “System Sentences with Elicitation”, “Control Button on Steering Wheel” and “Partial System Interface”. In addition, the results also indicate that the modified FMEA method is feasible and proposed as a systematic technique to assess the task of controlling the in- vehicle speech interaction system while driving. Through risk assessment and improvement for the potential problems, the drivers therefore may experience the enhancement of operation efficiency and driving safety.
ACKNOWLEDGEMENTS
Partial of this study was supported by Advanced Manufacturing and Service Management Research Center at National Tsing Hua University through Toward World-Class Universities Project (ICT Product Interface Design and System Innovation, Project Number: 101N7071E1).
REFERENCES
Barón, A., Green, P. (2006), “Safety and Usability of Speech Interfaces for In-Vehicle Task while Driving: A Brief Literature Review”, University of Michigan, Transportation Research Institute. Technical Report UMTRI 2006-5. Chen, F., Jonsson, I. M., Villing, J., Larsson, S. (2010), “Application of Speech Technology in Vehicles”, in: Speech Technology:
Theory and Application (pp. 195-219). Retrieved from: http://links.springer.com/chapter/10.1007/978-0-387-73819-2_11, DOI: 10.1007/978-0-387-73819-2_11 Fitch, G. M., Soccolich, S.A., Guo, F., McClafferty, J., Fang, Y., et al. (2013), “The Impact of Hand-Held And Hands-Free Cell Phone Use on Driving Performance and Safety-Critical Event Risk (DOT HS 811 757)”, Washington DC: NHTSA. Hua, Z., Ng, W. L. (2010), “Speech Recognition Interface Design for In-Vehicle System”, In Proc. 2nd International Conference
on Automotive User Interfaces and Interactive Vehicular Application, 11-12 November, 2010, Pittsburgh, Pennsylvania. Kmenta, S., Ishii, K. (2000), “Scenario-based FMEA: a life cycle cost perspective”, In Proc. ASME Design Engineering Technical Conferences, 10-14 September, 2000, Baltimore, Maryland. Lee, J. D., Caven, B., Haake, S., Brown, T. L. (2001), “Speech-based interaction with in-vehicle computer: The effect of speech-
based e-mail on driver’s attention to the roadway”, Human Factors Volume 43 No. 4. Lin, Q. L., Wang, D. J., Lin, W. G., Liu, H. C. (2014), “Human reliability assessment for medical devices based on failure modeand effects analysis and fuzzy linguistic theory”, Safety Science Volume62 No. 26. Liu, H. C., Liu, L., Liu, N. (2013), “Risk evaluation approaches in failure mode and effects analysis: A literature review”, Expert
Systems with Applications, 40(2), pp. 828-838. National Highway Traffic Safety Administration (2010), “Overview of the National Highway Traffic Safety Administration’s Driver Distraction Program”. Retrieved from: www.nhtsa.gov/staticfiles/nti/distracted_driving/pdf/811299.pdf Puente, J., Pino, R., Priore, P., de la Fuente, D. (2002), “A decision support system for applying failure mode and effects
analysis”, International Journal of Quality and Reliability Management, 19(2), pp. 137-150. Rhee, S. J., Ishii, K. (2003), “Using cost based FMEA to enhance reliability and serviceability”, Advanced Engineering Informatics, 17(3), pp. 179-188. Sellappan, N., Palanikumar, k. (2013), “Modified Prioritization Methodology for Risk Priority Number in Failure Mode and
Effects Analysis”, International Journal of Applied Science and Technology Volume 3 No.4. Shankar, N. R., Prabhu, B. S. (2001), “Modified approach for prioritization of failures in a system failure mode and effects analysis”, International Journal of Quality and Reliability Management, 18(3), pp. 324-336. Standard, S. A E. (2000), SAE J1739: Potential Failure Mode and Effects Analysis in Design (Design FMEA) and Potential
Failure Mode and Effects Analysis in Manufacturing and Assembly Processes (Process FMEA) and Effects Analysis for Machinery (Machinery FMEA), Warrendale, PA: SAE Ergonomics in Manufacturing (2020)
Strickland, D. L. (2012), “Visual-Manual NHTSA Driver Distraction Guidelines for In-Vehicle Electronic Devices”, Federal Register, 77(37), pp. 11200-11250. Teng, S. H. G., Ho, S. Y. M. (1996), “Failure mode and effects analysis: an integrated approach for product design and process
control”, International Journal of Quality and Reliability Management, 13(5), pp. 8-26. Yu, F. J., Hwang, S. L., Huang, Y. H. (1999), “Task Analysis for Industrial Work Process from Aspects of Human Reliability and System Safety”, Risk Analysis Volume 19 No.3.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in The Ergonomics in Manufacturing: Managing the Enterprise of the Future
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2103-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional