Raquel Marzo Roselló, Adrian Morales Casas, Vanessa Jimenez, Lia De Belda, Marta Valero, Mercedes Sanchís Almenara, Adrian Colomer Granero (2023). Agile and reliable design decisions based on the perception of the target audience. In: Usability and User Experience. AHFE Open Access, vol. 110, pp. 550–555. https://doi.org/10.54941/ahfe1003231
The conceptualization phase of a product plays a critical role in the design process. The decisions made during this phase directly affect the degree of innovation, quality of design solutions, costs involved, and overall competitiveness of the product. During this phase, a continuous process of accepting and rejecting options takes place, so that making decisions correctly becomes a very important activity for the company. However, decisions made during this phase often rely on subjective opinions, leading to uncertainty and errors. This paper presents the Waumap platform (https://waumap.ibv.org), that was developed as a methodological tool to involve potential consumers in the design process. This tool takes advantage of the potential of Artificial Intelligence (AI) to analyze in a standardized, agile and autonomous way the perception of the consumer expressed naturally by a representative sample of remote users, combining classical qualitative user research with natural language processing techniques.The paper shows also the results of the use cases carried out with different companies. In general, the companies state that the Waumap tool is very simple, intuitive and agile. They consider it very useful for their decision-making in their development processes, specifically, in the following application areas: new product concepts, supermarket shelf (physical and/or virtual), packaging and corporate image.The analysis of the perception of the consumers who have participated in the use cases have allowed to show that the automatic analysis of the user's perception through Artificial Intelligence reduces the time and effort of analysis to generate the keys that make it easier for companies to make decisions in the design of their products. The Waumap platform has positive implications for the decision-making process, facilitating a better integration of the target audience in the product conceptualization phase and offering the keys to design optimization from the point of view of market acceptance.
The conceptualization phase of a product plays a critical role in the design process. The decisions made during this phase directly affect the degree of innovation, quality of design solutions, costs involved, and overall competi- tiveness of the product. During this stage, companies continuously accept and
Agile and Reliable Design Decisions Based on the Perception 551
reject options, making it critical to make informed and appropriate decisions (Chaur Bernal, 2016).
However, decision-making during this phase largely relies on internal groups and collaborative sessions, which are highly subjective and influenced by the design team’s perceptions, tastes, knowledge, and previous experiences (Rasmussen et al., 1994). This often leads to uncertainty and errors during the design process, which is evident from the high failure rates of Fast-Moving Consumer Goods (FMCG) launches. Nielsen (2014) reports that 76% of FMCG launches fail in their first year of life, primarily due to the failure to address consumer needs or frustrations.
To address this issue, some companies conduct user research studies during the strategic definition phase to identify design requirements that cater to the needs of their target audience. However, these studies employ traditional market research methodologies such as surveys, focus groups, and interviews, which are time-consuming and expensive. Furthermore, these studies are con- ducted at the beginning of the process, limiting the flexibility to respond to market changes in later stages of development.
Moreover, marketing and user research managers in companies face con- straints in terms of time and resources (Bortels, 2019), resulting in a reduced frequency of these studies. Delaying involving potential consumers until the final validation stage increases the cost of development modifications signi- ficantly. According to Forrester (Souza, 2001), the cost of fixing a design problem after launch is 100 times higher than if it had been identified and rectified in the early stages of development.
When studies are conducted during the design phase, companies typically employ cost-effective and swift techniques such as surveys or AB tests. While these methods allow companies to decide on a design alternative among seve- ral options, they fail to provide information on the reasons for the preference or the related design elements, limiting the scope for maximizing customer satisfaction.
A NEW METHODOLOGICAL TOOL FOR INCLUDING POTENTIAL CONSUMERS IN THE DESIGN PROCESS To address the need for more efficient and effective methods for including users in the product development process, the Instituto de Biomecánica (IBV) has developed the Waumap methodological tool (https://waumap.ibv.org/). The tool utilizes Artificial Intelligence (AI) to analyze consumer perceptions of various design alternatives in a standardized, agile, and autonomous man- ner, allowing companies to make reliable and objective decisions during their design processes. The IBV has more than 40 years of experience providing advisory services to companies for designing target audience-oriented pro- ducts using the People Oriented Innovation (IOP) methodology (Instituto de Biomecánica, 2001). The Waumap study is conducted in three simple steps:
1. Defining the test based on two images and the characteristics of the target audience. The test comprises a survey (with indicators defined
552 Marzo Roselló
Figure 1: Waumap methodological tool to assist in decision-making during the design process.Figure 2: Waumap test AB eye tracking report.
by the company) and analysis of natural language using AI from open opinions expressed freely. Eye-tracking may also be incorporated. 2. Launching the remote study to a database of potential users that fit the defined profile. 3. Automatically generating a report, which can be received within 7 days of launching the study.
Agile and Reliable Design Decisions Based on the Perception 553
Figure 3: Example of design alternatives evaluated in the Waumap pilot study.
The Waumap report provides information on the factors driving preference and the emotions evoked by each design alternative through opinion polarity analysis. The eye-tracking feature generates heat maps displaying the areas of greatest visual attention through non-invasive webcam technology for gaze tracking analysis.
Here is an example of a Waumap report for design optimization from the perspective of market acceptance:
Table 1 provides a detailed description of the methodology for the auto- matic recording and analysis of user perception included in the Waumap tool.
To optimize the number of users needed to obtain reliable conclusions, a pilot study was conducted to estimate the effect size of the variable “favorite design alternative The appropriate formula was applied for experimental designs (contrast test) in which non-parametric statistical tests are applied and Cohen’s tables (Cohen, 1977) were consulted for a test power of 70% and a and a = 0.05, obtaining n = 50 users for the AB test and n = 75 users in the case of the AB test with eye tracking.
USER COMPANIES’ PERCEPTIONS OF THE WAUMAP TOOL FOR PRODUCT DEVELOPMENT The Waumap methodological tool has been validated through various use cases across different sectors, such as food, distribution, home, clothing, cle- aning, advertising, and tourism. These cases involved remote testing with representative user samples, which demonstrated the tool’s effectiveness in obtaining the keys to design optimization. User companies appreciate the tool’s simplicity, intuitiveness, and agility, especially in the areas of new pro- duct concepts, supermarket shelf (physical and/or virtual), packaging, and corporate image.
The Waumap results have provided valuable insights in the decision- making process, allowing companies to access a representative sample of users remotely and reduce costs in user recruitment, testing, and prototype development. Companies also appreciate the short period of time it takes to obtain results, with reports typically available within a week. The report’s
554 Marzo Roselló
Table 1. Waumap methodology.
Table available in the original PDF paper.
Recording techniques Analysis techniques Gaze tracking recording via webcam. IBV’s own programming using OpenCV Models to obtain areas of greatest visual attention. Recording of opinions expressed in a natural way, either by voice or text, to know the concepts perceived by the users of each design alternative, as well as the feelings generated by natural language processing.
- Amazon Transcribe. Amazon Web Services (AWS) automatic speech recognition service that facilitates speech-to-text conversion. It provides the transcribed text, as well as confidence scores between 0 and 1 (self-assessments by the service on how well it may have transcribed the text). It is used to convert the first impression expressed naturally by voice into text. - Amazon Comprehend. AWS natural language processing (NLP) service that allows, using machine learning techniques, to analyze and understand the meaning of text in different languages. Specifically, the Waumap tool uses the “Detect Sentiment” function, which analyzes the text and determines the polarity of the sentiment, i.e. whether the text is positive, negative, neutral or mixed. This function returns an inference of the predominant sentiment and provides for each sentiment the probability between 0 to 1 that it has been correctly detected. It is used to obtain the emotions of the first impression expressed in a natural way.
Calculation of the total % of participants with answers classified as positive grouped in Top 2 Box value, i.e., percentage of people who select for each survey concept the two most positive values.
Closed survey with concepts, to know the perception of the design alternative in some concepts predefined by the company. The ordinal Likert scale of 5 categories has been defined, as it is the most used scale in market research and easy to answer (Likert, 1932). In addition, the responses are coded from 1 to 5, with 1 being the value associated with the most disagreement (e.g., I do not like it at all) and 5 the value associated with the most agreement (e.g., I like it a lot), which makes it possible to quantify the evaluation of the design alternative.
Prioritization of design alternatives. Calculation of % of participants selecting and discarding each design alternative as favorite. Justification of the reasons for the prioritization of design alternatives.
- Natural Language Toolkit (NLTK). A set of Python libraries and programs that facilitate natu- ral language processing, such as the “nltk.corpus” library which provides access to a variety of cor- pora (text datasets). Among them is “Stopwords” which contains a predefined list of common words that do not provide value for text compression, such as articles, prepositions and conjunctions. - Wordcloud. Python function that generates a visual representation of the words in a text, where the size of each word represents its frequency. Prior to such visualization, natural language pro- cessing is required, such as the elimination of common words with the “Stopwords” function. In the Waumap tool, word clouds are represented for the selection and discard reasons for each design.
Agile and Reliable Design Decisions Based on the Perception 555
clarity and ease of understanding not only helps companies make decisions but also justifies them internally and to B2B clients.
The preference keys obtained through natural language processing analysis provide the most value in decision-making, enabling companies to under- stand the reasons for preference/rejection and their relationship with the design elements. Additionally, the results obtained with Waumap coincide with a qualitative study that the company conducted previously, but with a lower budget and in a shorter timeframe.
CONCLUSION
The automatic analysis of user perception through Artificial Intelligence reduces the time and effort required for analysis, enabling companies to make more informed decisions in the design of their products.
The Waumap platform has a positive impact on the decision-making pro- cess, facilitating better integration of the target audience in the product conceptualization phase and providing key insights into design optimization from the perspective of market acceptance.
ACKNOWLEDGMENT
The authors would like to acknowledge the Directorate General of Innovation of the Ministry of Innovation, Universities, Science and Digital Society, who financed the DEMOCRATIOP project (CONV21/DGINN/10). Also, we thank the following companies: Choví, Costa Brava, Pikolinos, Hosbec, SPB, Playfilm, Peronda and Luanvi. The development and validation of the Waumap tool would not have been possible without the active parti- cipation of these companies. Finally, we would like to thank José Antonio de Miguel Visa for his advice on the development of the Waumap tool using agile methodology and Lean Customer Development.
REFERENCES
Bortels, I. (2019). Estamos ante un ‘revival’ de la investigación cualitativa? I&A.
Investigación y Marketing. Chaur Bernal, J. (2016). Estudio sobre el proceso de toma de decisiones durante la fase conceptual del diseño de producto. Iconofacto. Cohen N. (1977) “Statistical Power Analysis for the behavioral Sciences”. Academic
Press, Inc. Instituto de Biomecánica (2001). Nuevas técnicas para el desarrollo de productos innovadores orientados al usuario. Nielsen (2014). Nielsen Breakthrough Innovation Report. Rasmussen et al. (1994). Cognitive engineering systems. New York: John Wiley &
Sons. Souza, R. K. (2001).”Get ROI from Design”. Cambridge: Forrester Research.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Usability and User Experience
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-86-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional