INTRODUCTION
Conceptual engineering provides a new way of exploring the conceptual foundations of human–technology interaction (HTI) research and design thinking (Chalmers, 2020; Floridi, 2011). This discipline enables people to modify traditional practices, which easily limit the scope of research.
In particular, cognitive ergonomics has been closely linked with cognitive psychology. One of the most important and successful theoretical idea transferred from it has been seeing human beings with limited capacity channels and focussing on mental workload (Broadbent, 1958; Miller, 1956).
The most important limitation of capacity-based psychological thinking is its inability to express mental contents, i.e. information contents of human mental representations. For instance, a telephone wire, which was the intuitive metaphor for the human mind in early cognitive psychology, cannot express information contents as it can be filled with messages having any contents (Shannon and Weaver, 1949). Questions related to analysing what people think and determining whether these thoughts are true or false has no place in studying mind as limited capacity channel (Saariluoma, 1997).
Capacity-based thinking can be enriched by system of theoretical concepts, which is meant to investigate and operate with the notion of “mental content” (Myllylä and Saariluoma, 2022; Saariluoma, 1995). A practical
© 2024. Published by AHFE Open Access. All rights reserved. 220
Problems of Content-Based Cognitive Ergonomics 221
application for content-based cognitive research is the development of intelligent technologies in industries.
In content-based thinking, mental contents are taken as representational information contents, and the properties of these contents —once analysed— are used to explain human actions (Myllylä and Saariluoma, 2022; Saariluoma, 1995, 1997). The difference between capacity and content can be seen in studying what kinds of questions these two ground concepts can express. On the one hand, capacity research is based on creating tasks requiring more processing capacity than what people can process and subsequently analysing the nature of errors. In memory research, subjects are, for example, submitted under long lists of stimulus elements or proven to have limits in human information processing capacity by means of secondary tasks (Baddeley, 2012; Miller, 1956).
Focussing on mental contents makes it possible to ask and answer new types of questions. For example, one can discuss truth and validity of representations: while it is possible for pilots to fly upside down, the question is not about their processing capacity but the validity of the information contents in their mental representations. Unless one has explicated the information contents, it does not make sense to ask whether their representations are correct. Contents are also necessary in understanding whether people use technical tools in a right way.
It may be difficult to see the function of mental contents in ergonomics, as the space of the needed scientific knowledge is apparently sufficient. However, in terms of mental contents, many classic concepts can be inspected from a new perspective. For example, “reliability” can refer to a system that works as it is supposed to work, which means the system is fault tolerant. However, from a content-based perspective, one can ask why we have good reasons to believe so. We cannot ask such a question in capacity-based thinking. Similarly, “trust” is a concept where you can ask its rationality on the grounds of reasons you have for trusting something or someone (Saariluoma, 2020). Thus, determining contents and ascertaining their truth are vital in working with such conceptual spaces.
While content-based thinking is relevant in basic research, it also plays a role in working with technology design. Today, technology design concerns not just technical artifacts but the human, organizational, and even social issues that are intimately linked to one another in developing intelligent technologies and intelligent societies (Fukuyama, 2018; Goede, 2020). The holistic nature of modern intelligent technology design gives justification to the presupposed role of mental contents in cognitive ergonomics and future design thinking.
A future problem for content-based ergonomics involves people working with intelligent technologies and controlling work processes. Combining machine and human intelligence demanding work processes in which human work is replaced with intelligent systems is a design goal for developing intelligent technologies. In the current paper, we investigate the control processes that occur in an intelligent control room. By interviewing employers, we can obtain knowledge about the structure of the process and describe it properly, which is the goal of our research.
The task of analysing intellectual work processes aims to obtain information regarding the designers and users of intelligent machine solutions and the processes they should be able to perform. Ships have sailed over a thousand years from Helsinki to Stockholm. Thus, there is an information process in the human mind that can carry out this process and is mainly guided by tacit information. As such, the challenge to psychology and cognitive science is to explicate such a process in the human mind. The process of sailing a ship from one harbour to another opens and presents opportunities for designers to mimic how people navigate ships and construct intelligent systems, either to help people navigate or replace people in those tasks (Saariluoma, Karvonen, and Sorsamäki, 2022; Wahlström et al., 2019). Our goals are to define and demonstrate methods as well as to illustrate how to use content-based data in creating technical solutions for practical problems. In the first stage, we collect qualitative data guiding human actions. Then, we investigate its contents and focus on qualitatively important aspects in the data. Finally, we suggest a solution for how to carry out the task at hand in the best possible way.
DEMONSTRATION OF CONTENT-BASED APPROACH IN ANALYZING INTELLECTUAL WORK PROCESSES IN HUMAN MIND To illustrate content-based approach and methods we have used material that was collected in semi-structured interviews from six people working with the remote operation and control of paper industry-related processes. Three interviews were carried out on-site, and three were conducted remotely. All participants gave their consent before the interviews.
How to Investigate Processes and Contents of the Mind Theoretically our analysis is based on a content-based cognitive scientific thinking (Myllylä and Saariluoma, 2022; Saariluoma, 1995). We wanted to obtain knowledge about the structures of the intellectual work processes and information contents of mental representations involved in paper industry. First, the main work processes and their subprocesses performed by the paper industry operators were recognized from the interviewees’ descriptions. In the second iteration, different thought processes regarding selected work tasks were identified, and the third iteration delved into the types of information content.
The following extract illustrates a remote operator’s narration of how they found the root cause of a particular problem case. The story begins with a description of the situation in which the process of root cause analysis started: “[Cognitive processes: Describing the frame of reference and building situational awareness] The power grid of the plant was completely cut off, and as a result [...], the automation system also crashed because the power outage was so long. [Understanding causalities, sense-making explanation, or end state] And then, of course, from the time the automation system is down, there are no data because nothing can be recorded. (Types and properties of contents: power grid, plant, result, automation system, crashing, power outage, long, from the time, data, and recording).
Problems of Content-Based Cognitive Ergonomics 223
[Describing the normal situation / ideal] When the boiler goes down, so that the places there do not heat up too much, then the feedwater pumps rotate the water to keep the pressure in order. [Reasoning with supporting premises] They are electrically operated, which is what is normally used. Then, there is the diesel-powered backup power pump in case electricity is lost, as happened in this case. [Identifying anomaly / exception] That spare pump did turn on, but it was just driving at some half power at a constant speed and not adjusting anything, [Identifying the problem to be solved / exception] then the question became, ‘Why didn’t it do anything but drive at that constant speed?’ (Boiler, going down, places (in boiler), heating up, too much, feedwater pumps, rotating the water, pressure, in order, electrically operated, normal, diesel-powered, backup power pump, this case, spare pump, driving, half power, a constant speed, adjusting, and question).
[Problem solving, identification of information gaps, knowledge of where to obtain missing information, and finding correction] Then I asked […] the commissioning crew […] for help, [Reasoning the root cause of the problem, a sense-making explanation or end state] and of course, the end result is that even if that pump starts to turn on, there is no control from the automation system [...]; it just spins at some constant speed that happens to be there. (I, asking, the commissioning crew, help, end result, pump, control, automation system, spinning, a constant speed, and happens to be).
[Estimating the significance or rarity of the situation, sense-making explanation, or end state] This is an exceptional [case] anyway, because that electricity did not go from the plant but from the entire power grid in that area. [...] [Reducing uncertainty, correction to the exception, and future thinking]. It was more like finding out what this problem was for the future. (Exceptional, electricity, plant, entire, power grid, area, finding, problem, and future).”
Several cognitive processes can be recognized from the operator’s speech in the above examples. These include describing the frame of reference as the person is building the foundations for situational awareness, understanding causalities, describing a normal/ideal situation and identifying an anomaly or an exception from that ideal, using additional premises to support conclusions, identifying the problem to be solved and the corrective actions to be taken toward the solution, identifying information and knowledge gaps and where to obtain such information, estimating the significance of the particular situation, creating a sense-making explanation about the problem situation and its root causes, and finally, reducing uncertainty if the exception occurs in the future. Notably, the types of content may consist of concepts (e.g., “power grid,” “feedwater pumps,” and “end result”), properties (e.g., “long,” “constant speed,” and “exceptional”), and actions (e.g., “driving,” “spinning,” or “finding”).
Somewhat similar but also different cognitive processes and contents can be identified in another extract, this time from a designer of process control solutions, who explains how operators use their displays after the implementation of certain design solutions: “[Cognitive processes: Describing the frame of reference and building situational awareness] What I do know is that they have this display in
the control room all the time. [Reasoning with supporting premises] Or at least that’s how they’ve commented to us and followed it daily, that we have received comments that they’ve clearly followed and [Explaining exception] seen the changes and have been able to say [Explaining system knowledge] that this calculation of ours is updated here every 10 minutes. (Types and properties of contents: I, knowing, they, display, control room, all the time, commenting, us, following, daily, comments, seeing, changes, calculation, ours, updating, and every 10 minutes).
[Reflecting how others perceive and explaining exception] They’ve seen the change that some of their metrics change, [Comparing to one’s own situation] but it’s not showing up here yet, [Reflecting how others reason] and then they’ve kind of figured out that [A sense-making explanation or end state] ‘OK, this updates with a certain frequency.’ (They, seeing, change, metrics, showing up, here, yet, figuring out, updating, and certain frequency).”
In this excerpt, the story also begins with a framing of the situation, followed by arguments and evidence to support the designer’s reasoning from both personal experiences (“That’s how they’ve commented to us”) and knowledge of how the system works (“This calculation of ours is updated here every 10 minutes”). Interestingly, the designer reflects how he thinks operators perceive and understand information when the situation changes, thereby illustrating a form of empathic thinking. In this example, there are properties of content (the personal pronouns “I,” “they,” and “ours”), mental processes (“knowing,” “seeing,” and “figuring out”), spatiotemporal properties (“daily,” “every 10 minutes,” and “here”), and concepts (“control room” or “metrics”), among others.
As the abovementioned examples illustrate, it is possible to investigate and identify the structures and properties of human intelligent processes and their contents by using a content-based cognitive scientific approach, which can also explain the rationale behind an individual’s judgments and actions. The mental processes and properties of the content that emerge depend on the imagined situation and the activities a person thinks about. However, we can already see from these two examples that some mental processes and contents are repeated in different contexts, such as setting a frame of reference, describing an ideal situation, thinking about corrective or supportive information and actions if there is an exception or change, and explaining things in a way that leads speakers (in their own opinion) to some reasonable conclusion. Similarly, content can include conceptual knowledge about things, such as devices, processes, or abstract phenomena, or may even be about spatiotemporal dimensions, actions of machines or systems, or behaviours or thoughts of oneself or others. Analysing mental contents can clarify, for example, the users’ and designers’ bases of thinking, their beliefs, and their construction of situational awareness. Such knowledge can help improve ergonomic design work and provides an important contribution to the literature on ergonomic design.
Problems of Content-Based Cognitive Ergonomics 225
DISCUSSION
The main application area of the presently described content-based approach to cognitive ergonomics is presumably in automatizing and autonomizing present-day work and life processes. Moving from data-driven Industry and Society 4.0 to human technology-driven Industry 5.0 and Society 5.0 strategies is often a process of moving from immediate human-controlled processes to their corresponding AI-controlled processes.
Content-based research is a new tool in the “toolbox” of cognitive scientists. In particular, the basic intuition behind such work is that people process information, and the content of such information is mainly responsible for how humans perform a task. As such, mental contents can also provide information about how machines could carry out this task. Today, content-based cognitive ergonomics concerns human interactions with intelligent technologies.
The term “content-based” refers to the use of identified mental contents in analysing and explaining human behaviours and actions. The content-based cognitive approach has been previously used to analyse human intellectual processes and mental contents in, for example, chess moves (Saariluoma, 1995), graffiti art experience (Myllylä and Saariluoma, 2022), and conspiracy thought models in climate change thinking (Myllylä, Cañas and Saariluoma, 2023). In the philosophy of mind, mental contents have been investigated by phenomenologists (Heidegger, 1926, Husserl, 1900–1901/2008). They have also been found to have an important impact on the analytic philosophy of language. For example, Wittgenstein (1953) and Fodor (1992) called attention to the analysis of mental contents. Several authors have discussed mental contents and their properties in phenomenological conscious experiences or in perceptual experiences of different sensory modalities (MacPherson, 2011; Montague, 2016; Siegel, 2017, 2021). Mental representations can also have non-perceivable content (Myllylä and Saariluoma, 2022; Saariluoma, 1992), such as information about social norms, values, and learned know-how in using an object or behaving in certain contexts, or mathematical or imaginary concepts that are different from sensory objects, although different types of contents can coexist, be related to each other, and combine in actual experience.
Among psychologists, the idea of mental contents has been a topic of research in clinical psychology. An important achievement in content- based thinking is Aaron Beck’s (1976) work on emotional processes in psychotherapeutic sessions, wherein data about mental contents and processes were obtained from verbatim recordings of subjects’ verbal descriptions and then evaluated to determine their validity against reality and reliable knowledge. The processing of conceptual semantic contents has been discussed in the psychology of spreading activation in memory networks (Collins and Loftus, 1975).
Content-based thinking has thus made many positive contributions to our understanding of the mind. However, it is important to present some general principles for content-based psychological thinking and their
applications in understanding human–machine relations and solving suitable problems in cognitive ergonomics. Notably, content-based thinking does not contradict traditional capacity-based ergonomics. Rather, they are just different perspectives on the problem of ergonomics. Both can be applied to different problems and for different purposes. While capacity-based thinking is used to analyse and explain how HTI processes surpass the limitations of human processing or why people commit errors in highly complex information processing environments, content-based thinking asks what kinds of mental contents we can discern and understand from human performance while controlling technological artifacts.
ACKNOWLEDGMENT
Saariluoma and Myllylä’s work was supported by the Etairos STN project of the Academy of Finland [Decision number 327355] and by the COACH project supported by Business Finland (Project number 3289/31/2022), operating under the umbrella of the SEEDForest ecosystem. We also thank the case companies and the participants in the COACH project for their input.
REFERENCES
Baddeley, A. (2012). Working memory: Theories, models, and controversies, Annual Review of Psychology Volume 63, 1–29. Beck, A. (1976). Cognitive therapy and the emotional disorders. London: Penguin
Books. Broadbent, D. E. (1958). Perception and communication. New York: Oxford University Press. Chalmers, D. J. (2020). What is conceptual engineering and what should it be?
Inquiry, 1–18. Collins, A. M., Loftus, E. F. (1975). A spreading-activation theory of semantic processing, Psychological Review Volume 82, No. 6, 407–428. Floridi, L. (2011). A defence of constructionism: Philosophy as conceptual engineering, Metaphilosophy Volume 42, 282–304. Fodor, J. A. (1992). A theory of content and other essays. Cambridge, MA: MIT
Press. Fukuyama, M. (2018). Society 5.0: Aiming for a new human-centered society. Japan Spotlight Volume 27, No. 5, 47–50. https://www.jef.or.jp/journal/pdf/220th_Spec ial_Article_02.pdf Goede, M. (2020). Society 5.0; We and I. University of Governance/ Miguel Goede/
Think Tank 3.0. Available at: https://www.academia.edu/41912670/Society_5_0_ We_and_I (Accessed: 8 April 2024). Heidegger, M. (1992). Being and time [Sein und Zeit]. 7th edn. Oxford: Blackwell.
(Original work published 1926). Husserl, E. (2008). Logische unterschungen I–II [Logical studies I–II]. V. Mayer & C. Erhard (Eds.). Berlin: De Gruyter. (Original work published 1900–1901). Macpherson, F. (2011) “Introduction: The admissible contents of experience”, in:
The admissible contents of experience, Hawley K. and Macpherson F. (Eds.). Chichester: Wiley-Blackwell, pp. 1–15. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information, Psychological Review Volume 63, No. 2, 81–97.
Problems of Content-Based Cognitive Ergonomics 227
Montague, M. (2016). The given: Experience and its content. Oxford: Oxford University Press. Myllylä, M. T., and Saariluoma, P. (2022). Expertise and becoming conscious of
something, New Ideas in Psychology Volume 64, Article 100916. Myllylä, M., Cañas Delgado, J. J., and Saariluoma, P. (2023). On conspiracy thought models in thinking climate change, European Journal of Sustainable Development Volume 12, No. 3, Article 15. Saariluoma, P. (1992). Errors in chess: Apperception-restructuring view, Psychological Research, Volume 54, 17–26. Saariluoma, P. (1995). Chess players’ thinking: A cognitive psychological approach.
London: Routledge. Saariluoma, P. (1997). Foundational analysis: Presuppositions in experimental psychology. London: Routledge. Saariluoma, P. (2020) “User psychology of emotional interaction—Usability, user experience and technology ethics”, in Emotions in technology design: From experience to ethics, Rousi, R. Leikas, J. and Saariluoma P. (Eds.). Human–computer interaction series. Springer, Cham, pp. 15–26. Saariluoma, P., Karvonen, A., and Sorsamäki, L. (2022). Human digital twins in
acquiring information about human mental processes for cognitive mimetics, Information Modelling and Knowledge Bases XXXIII Volume 343, Article 163. Shannon, C. E., and Weaver, W. (1949). The mathematical theory of communication.
Urbana, IL: University of Illinois Press. Siegel, S. (2017). The rationality of perception. Oxford: Oxford University Press. Siegel, S. (2021) “The contents of perception”, in; The Stanford encyclopedia of philosophy (Fall 2021 Edition), Zalta E. N. (Ed.). Available at: https://plato.stan ford.edu/archives/fall2021/entries/perception-contents/ (Accessed: 9 April 2024). Wahlström, M., Forster, D., Karvonen, A., Puustinen, R., and Saariluoma, P.
(2019) “Perspective-taking in anticipatory Maritime navigation-implications for developing autonomous ships”, 18th Conference on Computer and IT Applications in the Maritime Industries, COMPIT’19. Hamburg: Technische Universität Hamburg-Harburg, pp. 191–200. Wittgenstein, L. (1953). Philosophical investigations. Oxford: Basil Blackwell.