Stefano Filippi (2023). Relationships among Personality Traits, ChatGPT Usage and Concept Generation in Innovation Design. In: Artificial Intelligence, Social Computing and Wearable Technologies. AHFE Open Access, vol. 113, pp. 60–68. https://doi.org/10.54941/ahfe1004179
The literature reports many evidences about the influence of personality on design activities. At the same time, Natural Language Processing - NLP - tools are gaining importance day by day in product innovation. This research investigates possible relationships among personality traits, ChatGPT usage and the generation of innovative design ideas. A Microsoft Excel workbook implementing the first release of a data analysis framework has been developed and is available for downloading. The reader can use it to carry on personal evaluations; in the near future, an updated release of the framework will allow sending the results to a cloud repository to build a large database and perform more robust statistical analyses. This will allow the relationships highlighted up to now gaining objectivity and discovering new ones.
The demand for innovative and user-centric product design is growing, and so is the need for effective idea generation methods. In recent years, natural language processing (NLP) tools, such as ChatGPT by OpenAI and Bard by Google, have emerged as promising solutions for supporting idea generation in various domains (Füller et al., 2022; Bal Ram and Pratima Verma, 2023). At the same time, the literature already highlighted that specific personalities are better than others at suggesting innovative design solutions related to specific features, services and/or benefits, all of this referring to the product augmentation concept (Colgate and Alexander, 2002).
This research investigates possible relationships among personality traits, ChatGPT usage and the generation of innovative design ideas, by develop- ing a framework and testing it using data from a design experience. This framework, implemented in a Microsoft Excel workbook, is available to the readers in order to carry out personal analyses and evaluations. In the future, the results will be sent, anonymously and with the readers’ consent, to a common repository where data will be processed and used to strengthen the objectivity of the relationships already found and discover new ones.
The document structure runs as follows. The Material and Methods Sec- tion describes the data used to develop the framework and the way they
Relationships Among Personality Traits, ChatGPT Usage and Concept Generation 61
have been collected some months ago. Moreover, it reports some informa- tion about PERSEL, a tool developed some years ago to consider personalities in forming effective design teams. The Activities Section starts by analysing the data available and carrying on with the description of the framework development. It ends with the validation of the framework and by listing some first alleged evidences about the relationships. Some conclusions and research perspectives close the paper.
MATERIAL AND METHODS Available Data
The research described in the paper exploits the data collected during a design experience conducted in the academic year 2022–23 at the University of Udine, Italy, during the lessons of “Product interaction and innovation” in master’s degree courses in mechanical and management engineering. Some design tools and methods were available to foster ideas while developing an innovative product (Filippi, 2023). The experience occurred by following what suggested by the double diamond process model (Gustafsson, 2019). The starting point was the functional scheme of the product to design and the expected result was the morphology table filled with the concepts generated (Ullman, 2010). Each row of the table contains the concepts implement- ing a different product function. Figure 1 shows qualitatively one of the morphologies (concepts are in Italian).
Figure 1: One of the morphologies containing the concepts generated (in Italian).
In the meantime, students’ personality traits were collected using the BFI – Big Five Inventory – questionnaire (Goldberg, 1990; Sung and Choi, 2009).
62 Stefano
Figure 2 shows qualitatively the table collecting the answers of the partici- pants to the questionnaire.
Figure 2: Answers of the participants to the BFI questionnaire.
The logs of ChatGPT usage were recorded as well. Figure 3 shows an example of dialogue between a participant and ChatGPT.
Figure 3: Example of ChatGPT usage of one of the participants.
The performances of the participants were evaluated quantitatively using four well-known metrics: quantity, usefulness, novelty and variety (Sarkar and Chakrabarti, 2011; Filippi and Barattin, 2019). These metrics allowed comparing the solutions proposed; in other words, they allowed quantifying the impact of ChatGPT on the design activities (Figure 4).
A data management tool was developed, consisting in a Microsoft Excel workbook, to be used year after year to confirm the results of the research and to monitor evolution trends (Filippi, 2023). Figure 5 shows the interface
Relationships Among Personality Traits, ChatGPT Usage and Concept Generation 63
of this tool, named ChatGPT impact eval - Data analysis. The interface repli- cates the procedure to perform the data management; this helps a lot from the usability point of view.
Figure 4: Concepts have been classified and evaluated thanks to four well-known metrics. This allowed to evaluate the impact of the ChatGPT usage on design activities.Figure 5: The interface of the data management tool.
PERSEL PERSEL (ready-to-use PERsonality-based user SELection tool) has been developed some years ago (Filippi, 2020) to categorize people by the “big five” personality traits (Goldberg, 1990) and select the best candidates for a design team given the product features to focus on in innovation activities. PERSEL consists of a usable Microsoft Excel workbook where the interface, depicted in Figure 6, allows the input of the product features of interest (1), the printing of the BFI questionnaire (2), and the input of the participants’ answers (3). After that, the “RUN PERSEL” button makes the computation start and the result consists of the people to involve and how to involve them (5), the satisfaction of the conditions (product features to focus on) (6) and the percentage of the people involved (7). PERSEL has been exploited in this research for the management of the BFI questionnaire and the elaboration of the data collected thanks to it.
64 Stefano
Figure 6: The interface of PERSEL.
ACTIVITIES Data Analysis
In summary, three pieces of information were available at the beginning of the activities aimed at developing the framework: a collection of concepts, the outcome of the BFI questionnaire and the logs of the ChatGPT usage. The concepts have been used as they were, since they had been already elaborated and quantified thanks to the four metrics. Regarding the personality traits, simple formulas taken from PERSEL allowed to compute the big five for each participant, starting from the answers to the questionnaire. Figure 7 shows the result of this computation.
Figure 7: Personality traits of the participants to the design experience.
The analysis of the ChatGPT logs highlighted interesting elements from two points of view. The first one focused on the interaction type. Apart from the situations where Chat GPT had not been exploited (no interaction), par- ticipants interacted in three different ways. They ranged from looking for confirmation of self-developed concepts (interacting for validating hypothe- ses) to asking for solutions about specific product features (interacting to get hints about something specific), up to writing the entire problem to solve and asking for any sort of help (interacting to give ChatGPT the helm). All of this have been translated into the four interaction types as follows.
IT0: no interaction;
IT1: interacting for validating hypotheses;
Relationships Among Personality Traits, ChatGPT Usage and Concept Generation 65
IT2: interacting to get hints about something specific;
IT3: interacting to give ChatGPT the helm.
This information, in order to make the data as homogeneous as possible and their analysis the simplest, has been normalized. The interval [0,1] ranges from no use of ChatGPT to the request of its full help.
The second point of view regarded the empathy degree perceived during the interaction. The participants, with the supervisors’ help, evaluated and classified it as cold, impersonal, unfeeling rather than neutral, up to warm, highly empathic. Again, three labels have been associated to these different empathy degrees perceived during the usage of ChatGPT. They are as follows.
ED1: cold, impersonal, unfeeling;
ED2: neutral;
ED3: warm, highly empathic.
Again, for the same reasons as before, this piece of information has been managed as normalized. The zero value corresponds to the totally unpleasant empathy while one represents the perception of full warmness and friendliness.
Development of the Framework The framework has been implemented in a Microsoft Excel workbook. Par- ticular attention was placed on its usability, since the aim is to disseminate it to collect as many pieces of information as possible in a cloud repository. This way, the research results will be able to gain objectivity and consistency as time goes by.
Figure 8 shows the interface of the workbook. It lists all the actions needed to feed the database in order to get the relationships among the variables involved. Both the step descriptions and the images are hyperlinks to the workbook sheets of interest time by time.
Figure 8: The interface of the Microsoft Excel workbook implementing the framework to highlight the relationships among personal traits, ChatGPT usage and generated concepts.
66 Stefano
Although the interface should be auto explicative enough, the required steps to use the workbook are summarized in the following.
1) Collect data. The workbook allows printing the BFI questionnaire (1.1) and the morphology table (1.2), the material needed to collect data dur- ing the design experience. Moreover, it reminds to collect the logs of the participants’ interactions with ChatGPT (1.3). 2) Insert data. Collected data must be inserted in the workbook. The answers of each participant to the BFI questionnaire (2.1) as well as the generated concepts by all the participants (without doubles) (2.2). 3) Evaluate data. Workbook users are required to evaluate each concept from the usefulness and novelty points of view (3.1). This occurs based on the users’ knowledge and experience. As well, the interaction type and the empathy degree of each interaction with ChatGPT must be evaluated by analysing the logs (3.2). The users of the workbook are not required to count the number of concepts generated by each designer to set Quan- tities and to assign the Variety values because the workbook compute all these values automatically. 4) Consider the results. Once all data have been inserted, the ANALY- SIS sheet of the workbook will contain the results of the automatic computation. The interface invites the users to take a look at them.
Validation of the Framework and First Alleged Evidences About the Relationships The upper part of Figure 9 (cyan table) shows the content of the ANALYSIS sheet of the workbook. It represents the first, coarse statistical data analysis implemented in the current release of the framework. The formulas in the cells compute the one-to-one correlations between the independent variables (PT1-PT5, IT and ED) and the dependent variables (Q, U, N and V). Thresh- olds to highlight values to focus on have been set to -0.4 and 0.4. These values are unusual (they are quite far from the limits −1 and 1); they have been set this way in order to test the workbook functioning and to get some outcome rather than to obtain meaningful results. The workbook highlights the val- ues inside the range automatically. This first release of the workbook does not use the multiple linear regression, which would be more suitable to anal- yse the relationships among all the variables, for two reasons. First, again, the goal for now was solely to setup a working, usable framework; second, available data were clearly insufficient to allow any serious evaluation using that statistical approach.
Using the available data, seven relationships appeared of some interest. For example, it seems that the number of concepts generated (Quantity) increases according to the Interaction Type (IT). In other words, a designer who asks for the solution of the problem as a whole to ChatGPT will likely gener- ate more concepts than designers interacting with ChatGPT searching for just confirmations. The lower part of Figure 9, to the left, reports the scatter chart of the values involved in this example. The trendline has been added to highlight the positive slope of the relationship. The second scatter chart rep- resents the relationship between PT4 - Neuroticism and Usefulness. It seems
Relationships Among Personality Traits, ChatGPT Usage and Concept Generation 67
that neurotic designers generate concepts more useful than calm ones. Again, these results must be considered as scarcely meaningful since they come from a very small sample of data.
In summary, as depicted in Figure 9 (bold values), the alleged evidences highlighted thanks to the available data are as follows.
Quantity is inversely proportional to Neuroticism and directly propor- tional to Interaction Type and Empathy Degree;
Usefulness is directly proportional to Neuroticism and inversely propor- tional to Interaction Type;
Variety is inversely proportional to Agreeableness and Conscientiousness.
Figure 9: The results of the computation of the workbook with the scatter charts of two examples of relationships.
The reader can download the current release of the Microsoft Excel work- book implementing the framework here (https://uniudamce-my.sharepoint. com/:x:/g/personal/stefano_filippi_uniud_it/ESRSdwBHAgFNuusK9vYhb9Y BD3YYyN9zvLgiyNtBpJRf2w?e=UuyqA2).
CONCLUSION
This research aimed at highlighting relationships among participants’ per- sonality, ChatGPT usage and generated concepts, during innovation design activities. Some existing methods and tools have been exploited to reach the current situation, consisting in the availability of a framework for data analy- sis, fully implemented in a Microsoft Excel workbook, except for the module to send data to the repository for global elaboration. The dataset used to develop the framework, test it, and start highlighting some alleged evidences, is insufficient to say something definitive about the research results. Never- theless, it allowed to say that the framework works and is usable even by non-expert users, allowing in this way its dissemination to get more data.
68 Stefano
Data collection can already count on the involvement of NLP tools different from Chat GPT; Bard by Google and others will be kept into consideration.
Regarding some research perspectives, the availability of more data will allow refining the statistics by introducing the multiple linear regression anal- ysis. The module to send data to the cloud repository will be implemented and embedded in the new release of the framework. Other metrics to evalu- ate the generated concepts and to describe the interaction with the NLP tools will be considered, as well as other ways to qualify personality.
ACKNOWLEDGMENT
The author would like to acknowledge the students who participated to the design experience during the lessons of “Product interaction and innovation” in the master’s degrees in Mechanical Engineering and Management Engi- neering at the University of Udine, Italy, A.Y. 2022-23. The author is also grateful to prof. Ruggero Bellio, colleague at the University of Udine, Italy, for his help with the statistical analysis of this study.
REFERENCES
Bal Ram, Pratima Verma, 2023. Artificial intelligence AI-based Chatbot study of ChatGPT, Google AI Bard and Baidu AI. World J. Adv. Eng. Technol. Sci. 8, 258–261. https://doi.org/10.30574/wjaets.2023.8.1.0045 Colgate, M., Alexander, N., 2002. Benefits and Barriers of Product Augmentation:
Retailers and Financial Services. Journal of Marketing Management 18, 105–123. https://doi.org/10.1362/0267257022775927 Filippi, S., Barattin, D., 2019. Influence of Personality on Shape-Based Design Activities. Advances in Human-Computer Interaction 2019, 1–9. https://doi.org/10. 1155/2019/9651369 Filippi, S., 2020. PERSEL, a Ready-to-Use PERsonality-Based User SELection Tool
to Maximize User Experience Redesign Effectiveness. MTI 4, 13. https://doi.org/ 10.3390/mti4020013 Filippi, S., 2023. Measuring the Impact of ChatGPT on Fostering Concept Generation in Innovative Product Design. Electronics 12, 3535. https://doi.org/10.3390/ electronics12163535 Füller, J., Hutter, K., Wahl, J., Bilgram, V., Tekic, Z., 2022. How AI revolution-
izes innovation management – Perceptions and implementation preferences of AI-based innovators. Technological Forecasting and Social Change 178, 121598. https://doi.org/10.1016/j.techfore.2022.121598 Goldberg, L. R., 1990. An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology 59, 1216–1229. https://doi.org/10.1037/0022-3514.59.6.1216 Gustafsson, D., 2019. Analysing the Double Diamond Design Process through
Research & Implementation. Available online: https://aaltodoc.aalto.fi/handle /123456789/39285 (accessed on 6 September 2023). Sarkar, P., Chakrabarti, A., 2011. Assessing design creativity. Design Studies 32, 348–383. https://doi.org/10.1016/j.destud.2011.01.002 Sung, S. Y., Choi, J. N., 2009. Do Big Five Personality Factors Affect Individual Cre-
ativity? the Moderating Role of Extrinsic Motivation. soc behav pers 37, 941–956. https://doi.org/10.2224/sbp.2009.37.7.941 Ullman, D. G., 2010. The mechanical design process, 4th ed. ed, McGraw-Hill series in mechanical engineering. McGraw-Hill Higher Education, Boston.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Artificial Intelligence, Social Computing and Wearable Technologies
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-89-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional