Shion Matsuoka, Kimi Ueda, Hirotake Ishii, Hiroshi Shimoda (2025). Analysis of experimental consensus-building tasks with evaluation indices. In: Intelligent Human Systems Integration (IHSI 2025): Integrating People and Intelligent Systems. AHFE Open Access, vol. 160, pp. 97–102. https://doi.org/10.54941/ahfe1005814
We routinely engage in consensus-building activities on issues ranging from the trivial, such as deciding where to go on a group trip, to the socially significant, such as the issue of final disposal sites for radioactive waste. In order to build consensus, it is necessary to reach one unanimous conclusion [1].In such communication, it is known that “Kansei” plays an important role in improving human relations [2]. Therefore, it is believed that analyzing consensus building from the perspective of Kansei can provide suggestions for achieving more amicable consensus building.Therefore, in order to analyze the relationship between sensitivity and consensus building in the previous study, we designed a consensus building task to be used in an experiment to record the favorability for each utterance.In this consensus-building task, two experimental participants (A and B) decide by consensus “whether A will wait a certain amount of time without doing anything after the task ends.” B is initially provided with six foods, such as chocolate, etc. For A, waiting represents a penalty that A accepts.B uses these foods as bargaining chips. At this time, if both parties agree that A will not wait, both cannot get the foods; if A waits, both can get them as per the consensus distribution. However, the results of the consensus building were evaluated using the objective consensus results of waiting or not waiting and prize distribution only, and no significant relationship could be shown between the consensus building results and Kansei.It is said that gaining mutual trust is important in risk communication, which is a type of consensus building [3]. Therefore, we believe that it is possible to evaluate the results of consensus building from the perspective of trust in this consensus building, which discusses the acceptance of penalties.It is also said that there is a relationship between trust and the degree of agreement on the outcome. [3]Therefore, in order to analyze the relationship between Kansei and consensus building results, this study get the foods the level of mutual trust and the level of agreement with the results as subjective evaluation indices of consensus building results, and analyzes the relationship between them and Kansei.In other words, in order to investigate the degree of agreement and trust in consensus building, both parties are asked to answer the questions about their level of trust and their level of agreement with the results of consensus building after the task is completed.The experiment was conducted with 5 groups of 10 undergraduate and graduate students, and it was suggested that the sum of the trust level of both parties and the liking level have a relationship. We are currently running an experiment of the same content, and in the Final Manuscript, we would like to describe the results of this experiment in detail and specifically show its impact on the degree of trust and the degree of delivery in the consensus building process. Specifically, we would like to analyze the results of the consensus building process by focusing on the relationship between the ratio of the decrease in favorability of the proposing side and the level of agreement and trust.[1] Susskind, L., McKearnan, S., & Thomas-Larmer, J. (Eds.). (1999). The consensus building handbook: A comprehensive guide to reaching agreement, SAGE Publications, Inc.[2] Tei, S., Kawaguchi, T., Sim, T., Shizuka, H.(2020). Understanding and Supporting Users to Improve Atmosphere of Communication by Kansei Agents, Proceedings of International Symposium on Affective Science and Engineering 2020.[3] CLundgren, R. E., & McMakin, A. H. (2018). Risk communication: A handbook for communicating environmental, safety, and health risks (6th ed.). Wiley-IEEE Press.
Consensus building is defined as “a process in which a group reaches a unanimous decision on a proposition through communication” (Lawrence et al., 1999). In order to reach a unanimous conclusion from conflicting positions, the importance of not only rationality but also sensibility has been pointed out (Tei et al., 2020). This is because, as the framing effect demonstrates, even with a rational explanation, the outcome of consensus building can change depending on how it is presented or the situation, highlighting the limitations of rationality-based support for consensus building. Furthermore, in risk communication, which is a type of consensus building, trust and satisfaction—both subjective evaluations of the outcome—are considered important indicators (Covello et al., 2001). Particularly in consensus building that involves discussing the acceptance of disadvantage by some individuals for the sake of overall benefit, it is thought that the outcome can be evaluated based on trust and satisfaction. Therefore, by analyzing not only the objective evaluation of the consensus-building
outcome but also the subjective evaluation in terms of trust and satisfaction from the perspective of “Kansei” (emotional sensibility), it may be possible to gain insights into more harmonious methods of consensus building.
However, while many humanities scholars have analyzed consensus building based on linguistic information such as conversation content, few studies have observed and analyzed it from the perspective of sensibility (Hamada et al., 2019).
PURPOSE OF THIS STUDY In previous research (Matsuoka et al., 2024), a task was developed to analyze the consensus-building process from the perspective of Kansei. This task simulates the issue of constructing a waste disposal site, specifically recreating a situation where the nearby residents discuss whether or not to accept a disadvantage for both the residents and the government to gain mutual benefits. The task involves two participants (A and B), who use a chat application to decide whether A will wait for two hours after the task is completed, without doing anything. B is provided with six chocolates or other food items as negotiation tools.
For A, waiting represents the disadvantage that A must accept. If both agree that A will not wait, neither participant can take the food items home. However, if A agrees to wait, both participants can take the food items home according to the distribution decided during the consensus-building process. As a motivation for B to participate in the consensus, the waiting time is set based on the distribution of rewards.
As shown in Figure 1, the change in favorability toward the speaker for each statement was recorded as the true value (input by the listener) and the predicted value (input by the speaker) on a 9-point scale ranging from −4 to +4 using the chat application as a tool for measuring Kansei. However, in previous research, consensus-building was evaluated only based on objective outcomes, without evaluating the subjective trust in the other participant and satisfaction with the outcome. Therefore, this study aims to analyze consensus-building with a focus on Kansei, proposing trust in the other participant and satisfaction with the outcome as evaluation indicators for a harmonious consensus. The analysis specifically focuses on A’s (the one who accepts waiting) trust, satisfaction with the outcome, and changes in favorability toward B.
Figure 1: An example screen of text chat system (UserB).
Analysis of Experimental Consensus-Building Tasks With Evaluation Indices 99
EXPERIMENT In this study, the consensus-building process was recorded using the task developed in previous research and a chat application. This experiment consists of four steps. First, participants were given an explanation of the experimental task and the usage of the chat application. Afterward, they practiced using the chat to ensure that the actual and predicted values of changes in favorability (Kansei) were correctly recorded. The task mentioned above was then carried out. Finally, participants completed surveys on the harmony of the consensus, demographic information, and the perceived distance to reaching consensus.
In this study, to evaluate the harmony of the consensus-building results, satisfaction with the result and trust in the other participant were used as indicators. These indicators are considered important elements in evaluating consensus-building outcomes in risk communication. After completing the task, participants responded to a harmony evaluation survey via Google Forms, using a 9-point scale from −4 to +4. In addition, the demographic survey included questions on age, gender, and frequency of text chat usage. The participants’ cooperativeness was also measured using the shortened Big Five scale (Goldberd, 1981). Furthermore, the “distance to consensus”survey asked participants to evaluate the perceived distance to consensus on a 101- point scale, with 100 representing the start and 0 representing the completion of the consensus, based on the dialogue log.
The experiment was conducted between July 22, 2024, and October 23, 2024. The participants, 20 Kyoto University students and graduate students (10 pairs), were recruited through Kyoto University’s co-op part-time job recruitment system. All participants were native Japanese speakers and proficient in typing on a keyboard. They received compensation equivalent to 2,000 JPY for their participation. The experiment was conducted with the consent of all participants and was approved by the Ethics Committee of the Graduate School of Energy Science, Kyoto University.
RESULT
In the analysis of this experiment, data from 8 instances where both participants A and B engaged in more than 20 exchanges were analyzed. As a result, in 2 cases, A agreed not to wait, while in 6 cases, A agreed to wait.
Figure 2 shows the relationship between A’s trust in B and B’s favorability decrease rate. Eq. 1 shows the calculation of B’s favorability decrease rate.
Fd = Sd
St (1) Fd represents B’s favorability decrease rate, Sd is the number of statements made by B that resulted in a decrease in A’s favorability toward B, St is the total number of statements made by B. It was found that there is a possibility of a linear relationship between B’s favorability decrease rate and A’s trust in B. In other words, the higher B’s favorability decrease rate, the lower A’s trust in B may become. This suggests that, regardless of the outcome of
the consensus-building (whether A agrees to wait or not), the favorability decrease rate could potentially influence the level of trust.
Figure 3 shows the relationship between A’s satisfaction with the outcome and B’s favorability decrease rate. The analysis showed no clear linear relationship between B’s favorability decrease rate and A’s satisfaction. However, looking at the graph in Figure 4, which depicts the 6 cases where A agreed to wait, a linear relationship between A’s satisfaction with the outcome and B’s favorability decrease rate can be observed. In other words, it was confirmed that when A agreed to wait, the higher B’s favorability decrease rate, the lower A’s satisfaction with the outcome.
Figure 2: Participant B’s percentage decrease in favorability and trust from A to B.Figure 3: Participant B’s percentage decrease in favorability and satisfaction with the results of A.
Analysis of Experimental Consensus-Building Tasks With Evaluation Indices 101
Figure 4: Participant B’s percentage decrease in favorability and satisfaction with the results of A when A agrees to wait.
DISCUSSION AND CONCLUSION In this study, we proposed “trust in the other participant” and “satisfaction with the outcome” as subjective evaluation indicators of consensus- building results and analyzed the experimental data from the perspective of favorability based on these indicators. As shown in Figures 2 and 4, there is a possible correlation between favorability and trust in the other participant, as well as between favorability and satisfaction with the outcome. Both the final consensus outcome, representing rationality, and favorability are believed to significantly influence the subjective evaluation of consensus-building. Therefore, the correlation between favorability and trust suggests that trust in the other participant may be more strongly influenced by emotional factors than by rational ones. Regarding satisfaction with the outcome, after excluding the two cases where A did not agree to wait, a possible correlation between favorability and satisfaction was confirmed. This finding suggests that when the rational evaluation, such as the final consensus outcome, is the same, Kansei, or emotional factors, may influence satisfaction with the outcome. Furthermore, it was suggested that a decrease in favorability might have a stronger impact on trust and satisfaction than an increase in favorability, especially in cases where A agreed to wait. This indicates that a decrease in favorability may have a greater effect on the subjective evaluation of consensus-building than an increase.
However, since there were only two cases where A did not agree to wait, it is necessary to verify whether a correlation exists between favorability and satisfaction in such situations as well. In the future, by increasing the number of experimental cases, we aim to clarify the relationship between trust, satisfaction, and favorability. Additionally, since it was found that preventing a decrease in favorability is important for maintaining a positive subjective evaluation of consensus-building, we plan to analyze which types
of statements cause a decline in favorability. We will classify the statements according to their conversational function to examine how each type of statement affects the subjective evaluation of consensus-building.
REFERENCES
Goldberd, L. R. (1981). Language and individual differences: The search for universals in personality lexicons, Review of personality and social psychology, 2, pp. 141–165. Hamada, Y., Maruyama, T., Shoji, H. (2019). Pattern Classification of Value Creative
Consensus Building Process in Case of Multiple-choice, International Journal of Affective Engineering, 18(3), pp. 129–136. Lundgren, R. E., & McMakin, A. H. (2018). Risk communication: A handbook for communicating environmental, safety, and health risks (6th ed.). Wiley-IEEE Press. Matsuoka, S., Kamakari, R., Ueda, K., Ishii, H., Shimoda, H. (2024). Design of
a consensus-building experimental task considering the asymmetry of positions, Human Interface symposium 2024, pp. 274–280 (in Japanese). Susskind, L., McKearnan, S., & Thomas-Larmer, J. (Eds.). (1999). The consensus building handbook: A comprehensive guide to reaching agreement, SAGE Publications, Inc. Tei, S., Kawaguchi, T., Sim, T., Shizuka, H. (2020). Understanding and Supporting
Users to Improve Atmosphere of Communication by Kansei Agents, Proceedings of International Symposium on Affective Science and Engineering 2020.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2025): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-36-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional