Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
INTRODUCTION
We have presented in recent publications (Laurentiz 2019, 2021) different approaches and uses for artificial intelligence algorithms in artistic works. Some algorithms can find statistical patterns in large amounts of data and generate images that simulate representational models of different natures. The nonconformist artist does not just apply formulas, operations, calculations, functions, and models. Their commitment is different. The artist can intercede in the process itself, subverting the very function of modeling and algorithm, denouncing this conformed way of producing images. In this sense, is interesting to understand which layers and procedures are being questioned and transgressed by the artist's poetics. Understanding different representational systems and their procedures opens a field of possibilities. This work intends to present some studies already carried out on this subject and to point out some consequences for language and thought. The main interest is to present how art can contribute to the process and development of computational algorithms - in particular, to artificial intelligence. Just as the human mind has specific strategies and capabilities, there are non-human abilities that computational algorithms possess, and both mind and algorithm bring them into this relationship, into an engagement of creative action. According to Jorge Albuquerque Vieira, the artist explores fields of possibilities of his/her surroundings (Umwelt) and ends up perceiving sophisticated articulations of reality that follow criteria of organization and coherence, which are associated with an 'aesthetic root' (Albuquerque Vieira, 2010). The Aesthetic behavior is "[...] an integral part of human modeling that has enabled humans not only to negotiate the complexity of their environment more ably, but also to envisage new (aesthetic) worlds in a manner which is not identical to, but is cognate with, attempting to anticipate the future” (Hoffmeyer 2015, p.157).The justification for this work is that we can no longer underestimate technology as a mere instrumental manifestation; much less believe in its supposed neutrality, since technology brings the dominant discourses embedded in it (Hui 2020). In the words of Ivo Ibri, "[...] simply contemplating the world, in a disinterested experience because it has no practical purpose, allows us to demobilize the conceptual forms that mediate our acting in the world” (Ibri 2020, p.6). This makes the artist and art fundamental pieces of this puzzle of conformed thoughts, representational systems, modeling, and creative cognitive procedures.
The purpose of this article will be to show procedures and representational models used by artists in a poetic and creative way, accompanied by some studies by theorists from different areas that will corroborate with the proposal. This is speculative research intends to understand which layers and procedures are being questioned and transgressed by the artist's poetics. The premise is that understanding these representational systems and their procedures opens a field of possibilities both for the artist's experiences and for the computer scientist in the development of their programs, in intertwining of creative actions.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
ART’S CONTRIBUTIONS FOR ARTIFICIAL INTELLIGENCE RESEARCH In this sense, the artist can, for example, train a machine and make it ‘lose’ on purpose, what it has learned (a nonsense for a scientist), reminding us that in the human capacity remembering and forgetting are accomplices and not adversaries in the thought process. This is the case with the video work entitled What I Saw Before Darkness (AI Told Me, 2019, at https://vimeo.com/337909277, accessed September 2021) by an artist who simply goes by the name "the girl who talks to AI".
The artist can also create a system that feedbacks actions from agents of living systems and artificial systems, in a hybrid process capable of triggering improvisations, mishaps, accidents, and new stimuli during performances, including imperfections to the process, as the case of the work Degenerative Cultures by Cesar & Lois. The authors call it "A Post- Anthropocentric Intelligence" and explicitly are "Corrupting the Algorithms of Modern Societies" (Cesar & Lois, 2018, at https://cesarandlois.org/digitalfungus/, accessed September 2021). These strategies are important escape valves for the pitfalls of representational systems with goal-directed programmed behavior. The aesthetic outcome of all these procedures will then be relearned, re-evaluated, and reconfigure experiences through circularity and systemic feedback.
Just as the human mind has specific strategies and capabilities, there are non-human abilities that computational algorithms possess, and both mind and algorithm bring them into this relationship, into an engagement of creative action.
Aesthetic behavior, “[...] an integral part of human modeling that has enabled humans not only to negotiate the complexity of their environment more ably, but also to envisage new (aesthetic) worlds in a manner which is not identical to, but is cognate with, attempting to anticipate the future” (Hoffmeyer 2015, p.157).
This seems to agree with Albuquerque Vieira, who says that the artist explores fields of possibilities of his/her surroundings (Umwelt) and ends up perceiving sophisticated articulations of reality that follow criteria of organization and coherence, which are associated with an 'aesthetic root' (Albuquerque Vieira 2010).
Shinseungback Kimyonghun is a Seoul-based artist duo consisting of Shin Seung Back and Kim Yong Hun, authors of the work Cat or Human (2013, at https://ssbkyh.com/works/cat_human/, accessed July 2021). The work is composed of two sets of one hundred photographs that use human face detection algorithm (OpenCV`s) and detection for cat faces (KITTYDAR) in an inverted form. So, Flicker Photos were used, and the program recognized human Faces by the cat face detection algorithm, and cat Faces were
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 recognized as human faces by a human face detection algorithm. When the artist uses tools and techniques in unconventional ways and does not use them for the function for which they were created, they explore this field of possibilities that Albuquerque Vieira announced. In this case, the artist ironically denounces weaknesses and inaccuracies in the recognition system.
The classic book What computers can't do: a critique of Artificial reason by Hubert Dreyfus (Dreyfus 1972), already pointed out strong reasons for the difficulties of programming intelligent activities by the computer. One of them was the fact that the computer covers only mathematical thinking, which differs from perceptual thinking. The latter depends on a body inserted in a context and in an environment, as well as on indetermination, that is a characteristic of perception.
“We have seen that, as far as we can tell from the only being that can deal with such ‘vagueness’, a ‘machine’ which could use a natural language and recognize complex patterns would have to have a body so that it could be at home in the world” (Dreyfus 1972, p. 216).
Another issue that he said should be taken into consideration was the discrete nature of all computer calculations. There is also the fact that the human mind has flexibility and can perform creative actions to solve problems, and a machine does not (Dreyfus 1972). Already at that time, a proposal was made to think about systems that would promote a symbiosis between computers and human beings, because together they could accomplish things that could not be done separately.
Another point of view is that of Yuval Noah Harari (Harari 2018), who considers that AI is beginning to surpass humans in an increasing number in cognitive abilities such as learning, analyzing, and communicating, even going on to understand human emotions (Harari 2018, p. 37). Assuming that our decision-making is the result of billions of neurons calculating probabilities in a fraction of a second, that "human intuition" (Harari 2018, p. 38) is the ability to recognize patterns, and that emotions and desires are "no more than biochemical algorithms, there is no reason why computers should not decipher these algorithms” (Harari 2018, p. 39). The author argues that the brain recognizes biochemical patterns by analyzing facial expressions, voice tones, hand movements, and even body odors; and that an AI equipped with the right sensors could do all this with much more accuracy and reliability than a human. But the problem is not only in overcoming abilities that seemed to be exclusively human, but rather, in the fact that there are non-human abilities that would differentiate AI in a qualitative way in this process. According to Harari, connectivity and updateability would put us at a disadvantage in the future (Harari 2018, p. 39-40). In this sense, we reinforce the idea that human, and machines should work together.
The artist Sougwen Chung can offer us an important contribution. Her emblematic work ‘On the collaborative space between humans and non-humans’ (https://sougwen.com/on-the- collaborative-space-between-humans-and-non-humans, accessed July 2021) presents human-robotic performances that generate drawings collaboratively, and arise from the
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 relationship of a living system with an artificial one. The performances utilize one or several robotic arms, which respond to a variety of data inputs that the artist has been developing for some time. As the artist herself explains, in the beginning she used a gesture-based approach, using a neural network. The line drawing that the artist did manually was recorded in real time, either by an overhead camera or by a sensor on the tip of the brush and transformed its positional data into something that could be read by the system. The system acquired this data, articulated by the robotic units, producing a set of positions based on an interpretation of the artist's drawings. It is a process that is built from a feedback cycle with the drawings made during the performance, as if they were a true 'dance of graphic lines'. Over time, the artist has integrated other data into the system, such as her biometrics, heart rate, and brain waves. Her idea is to think about ways in which humans connect to mechanical and artificial systems, and vice versa, and which would function as a 'creative catalyst'. Several important questions arise from this work, we will highlight only a few.
First, the artist's traces are retrieved by devices (be they cameras or sensors) and the extracted data enables the interaction between the agents: the gesture and body of the artist and the program and the robotic arm performing drawings together. It is important to realize that, by passing this data through devices, we are dealing with third-degree abstraction (Flusser 2007). But more relevant is how the artist says that she ended up adapting to the "inaccuracies of the machine", which led her to readjust her own gestures. It is a paradox, since a machine would not have, in its nature, principles of im-precision. It is imprecise compared to the complexity of the drawing performed by the artist's gesture, which the mathematically precise calculation and the flexibility of the robotic interface cannot achieve. It is important to say that the AI was trained on a second moment from the artist's numerous drawings. At first, it was only the real-time data that was captured and processed during the performance, but then the system was implemented by a learning process from a set of 20 years of data retrieved from the artist, which reflected a certain "trend of her style" as she adapted to the movements of the robot, in continuous circularity. One can clearly see the feedback process between the two, and that is described in the project by the collaborative involvement of the creative action itself. This involvement, defined as a 'creative catalyst', causes significant changes, both sensory and cognitive, in the aesthetic experience.
Despite this increase in complexity, the results we are getting with current Machine Learning systems show that we will have to revise and update our current bases, and even generate new ontologies, from a more dense and complete set of descriptions about an image. Recall the ImageNet case and Microsoft researcher Kate Crawford's criticism that we are injecting our own limitations into the algorithms. In her article Excavating AI - The Politics of Images in Machine Learning Training Sets (Crawford 2019), she highlights that making a machine interpret images is much more of a social and political issue, rather than merely a technical one. ImageNet ultimately demonstrated how these processes can promote discrimination, misjudgments, biases, and that the technical process of categorizations and classifications proves to be a political act. There-fore, the production of images made by a machine carries
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 social, political, and economic issues based on the context in which they are inserted, their surroundings, and because of their condition as an "allopoietic system" (Nöth 2001, p.66).
The entire basis of machine learning systems is built on training sets, and it is these that underlie how the AI will recognize and interpret data from the world. Joy Buolamwini, who identifies herself as the Poet of Code (https://www.poetofcode.com/, accessed July 2021), working with facial analysis software, realized that the software could not detect her face because the people who coded the algorithm had not taught it to identify a wide range of skin tones and facial structures. After that, she faced a mission to combat bias in machine learning, the result of a "Coded Gaze".
Still according to Crawford, this is not an easy task since images are loaded with multiple senses and meanings. "Entire subfields of philosophy, art history, and media theory are dedicated to teasing out all the nuances of the unstable relationship between images and meanings" (Crawford 2019).
In this sense, it is also important to know the trajectory of the research that culminated with Deep Learning (Kurekov 2020), so we can understand how the generation of models in the computer takes place and have a better understanding of what a simple image is. In fact, according to Lev Manovich (Manovich 2019), the challenge is to try to go beyond the search for types, structures, and patterns from already existing and recognized ways of seeing the world.
An artwork that is very moving is Rafael Lozano-Hemmer's Level of Confidence (2015). It is a project that deals with the mass kidnapping of forty-three students from a school in Mexico. It was released exactly six months after the kidnapping. The project consists of a camera and a facial recognition algorithm, which relentlessly searches for the faces of missing students. When you stand in front of the camera, the system uses algorithms to find the students' facial features that most resemble you and gives a “confidence level” (http://www.lozano-hemmer.com/level_of_confidence.php accessed July 2021), which is a degree of matching accuracy in analyzing shapes in percentage. The artwork denounces that catastrophe saying that there are chances of you being one of those students.
The Series Fifty Sisters that Jon MacCormick presented at the 2019 Siggraph event, “is comprised of fifty computer synthesized plant-forms, algorithmically ‘grown’ from computer code using artificial evolution and generative grammars. Each plant-like form is derived from the primitive graphic elements of oil company logos” (https://digitalartarchive.siggraph.org/artwork/jon-mccormack-fifty-sisters/ accessed July 2021).
The title refers to the “Seven Sisters” – a cartel of seven oil companies that dominated the global petrochemical industry and oil production in the Middle East from the mid-1940s until the oil crisis of the 1970s. In the description of the work, it is commented that the motivation of the work was the fact that oil shaped our civilization and boosted its unprecedented growth
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 in the last century. We were seduced by oil and its by-products as they are now used in almost every aspect of human activity, providing fuels, fertilizers, raw materials, plastics, medicines and much more. But on the other hand, oil has also changed the environment, whether because of the petrochemical fog that hangs over many modern metropolises, or the environmental damage from major oil spills and the global climate crisis. The work poetically denounces the devastation that humanity's appetite for oil ends up causing to the environment. This leads us to a broader discussion by Yuk Hui, in his book on Technodiversity, "the end of unilateral globalization and the arrival of the Anthropocene force us to talk about cosmopolitics. These two factors are correlated and correspond to two different meanings of the word ‘cosmopolitics’: as a commercial regime and as a policy of nature" (Hui 2020, p. 16). As well as: 1. technology is not anthropologically universal; 2. its functioning is assured and limited by cosmologies that go beyond mere functionality and utility; 3. there is no single technology, but a multiplicity of cosmotechniques.
Finally, artist Anna Ridler brings a discussion of the machine learning model itself. In the work Mosaic Virus (2018, 2019, at http://annaridler.com/mosaic-virus, accessed July 2021), tulips are controlled by the price of bitcoins, showing the fluctuation of the financial market in a kind of technological cultivation of artificial tulips. The artist relates this work to a 17th century phenomenon called Tulip Mania, in which the price of tulip bulbs reached high values and then fell dramatically. This case has been used as an example of a speculative bubble, and the artist thus creates strong parallels with the current cryptocurrency speculation. There are several suggested relationships, from having the word virus in the title, which is the name of a disease that causes stripes to appear on plants, and here the artist creates an analogy with the behavior of bitcoins, as if they were a virus that would control the appearance of the flower, to the very current discussion about valuing art objects. There is also the fact that the virus makes the flowers weaker, guaranteeing only a certain number of tulips on the market with special characteristics, while in bitcoin, there is a finite amount of them on the market, and this makes them more attractive for in-vestments and speculation. At this point, what we would like to point out is that this work echoes in the very AI model used, reflecting on how GANs work. Therefore, the artist gives new direction to the machine learning system itself.
As the site of the work states: "The motion of the 'boom and bust' of the markets is also evident in the way that GANs work; As the model strives towards perfect encapsulation of the tulip, its collapse mirrors the ups and downs of speculative bubbles. When they are training, they sometimes tend to seem like they are improving - the learning rates will go up and up – and then suffer 'mode collapse' [...]" (http://annaridler.com/mosaic-virus, accessed July 2021).
Anna Ridler uses technology, and in particular machine learning, from how these algorithms help her conceptually construct her poetic proposals. And the GANs had interesting technical aspects to be addressed, but they were not just tools for the artist, as they brought another look at those interconnected subjects. This example becomes appropriate for the closing of
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 this text, because, by putting the AI itself in check, the artist demonstrates her concern and alerts us about the aesthetic and ethical responsibility we have for these systems.
FINAL CONSIDERATIONS There are so many questions that artists ask us when they use complex algorithms and AI that the scientist, carrying out her/his work for other purposes, fails to notice. I hope I have shown that artist and scientist should always work together. Mentioning Ivo Ibri once more, from a disinterested experience with no practical purpose, allows the artist to demobilize and obtain another point of view of the conceptual forms of the world (Ibri 2020, p.6). This makes the artist and art fundamental pieces of this puzzle of ‘conformed thoughts’ (Laurentiz 2019), representational systems, and creative cognitive procedures.
REFERENCES
Albuquerque Vieira, J (2009/10), “Teoria do Conhecimento e Arte”. Revista Música Hodie, 9(2). doi: 10.5216/mh.v9i2.11088 (accessed Dez 2021), 11-24. Crawford, K. and Paglen, T. (2019), “Excavating AI: The Politics of Training Sets for
Machine Learning”. The AI Now Institute, NYU, September 19, 2019, at https://excavating.ai/ (accessed Dez 2021). Dreyfus, H. (1972), What Computers Can’t Do: A Critique of Artificial Reason, Harper & Row, Publishers, USA. Flusser, V. (2007), O mundo codificado, Ed. Cosac Naify, São Paulo, Brasil. Harari, Y. (2018), 21 lições para o século 21, Companhia das Letras, São Paulo, Brasil. Hoffmeyer, J. (2015), “Semiotic scaffolding: a unitary principle gluing life and culture
together”. Green Letters Journal, 19:3, 1-12, DOI: 10.1080/14688417.2015.1058175 Hui, Y. (2020), Tecnodiversidade, Ubu (Translated by Humberto do Amaral). Ibri, I. (2020). Semiotics and Pragmatism – Theoretical Interfaces (Vol. 1), Editora Cultura Acadêmico, Marília, Brasil. Kurenkov, A. (2020), “A Brief History of Neural Nets and Deep Learning”. Skynet Today,
at https://www.skynettoday.com/overviews/neural-net-history (accessed Dez 2021). Laurentiz, S. (2021), “Art in the Context of Algorithmic Logic Procedures”. Arbor, 197(800), a603, at https://doi.org/10.3989/arbor.2021.800005 (accessed Dez 2021). Laurentiz, S. (2019), “Conformed thought and the Art of Algorithms”. Proceedings of the
9th Inter-national conference on digital and interactive arts - ARTECH, 23-25 October 2019, Braga, Portugal, ISBN/ISSN 9781450372503, p. 465-472. Editora Universidade Católica Portuguesa, Portugal. Manovich, L. (2019), AI Aesthetics, Strelka Press, Moscow, December 21, 2018, ASIN: B07M8ZJMYG, at http://manovich.net/index.php/projects/ai-aesthetics (accessed Dez 2021). Nöth, W. (2001), “Máquinas semióticas”. Revista Galáxia: Revista do Programa de Pós-
Graduação em Comunicação e Semiótica, n.1, São Paulo.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7