Gianmarco Longo, Deborah A Middleton, Silvia Albano (2024). Elaborating a Design Framework That Is Able to Structure and Guide Composition of Generative AI Visualizations. In: Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications. AHFE Open Access, vol. 120, pp. 286–295. https://doi.org/10.54941/ahfe1004579
One aspect of using generative artificial intelligence (AI) in design workflows that is not well understood, is how designers navigate the use of prompts and imagery to control design outputs. To learn more about the ways generative AI can elaborate different innovative design aspects linked to a conceptual genre, we undertake an analysis of designer outputs and link these to a mapping of iterative design workflow, and the composition of prompts and imagery used to inform generative AI visualizations. A speculative design workshop presented young designers with a proposed framework to facilitate and guide them through an iterative design process using generative AI tools (Vizcom and Midjourney). Design outputs included designed objects, atmosphere, spatial configurations able to generate social communication and relations for futuristic residential lighting environment. Prompt writing and sketching were two crucial variables that enable high fidelity design visualization. Traditionally research on prompt writing focuses on variables of descriptive design imagery, point of view, quality of output, ordering, precision, tone and style. (Microsoft,2023) In this study we investigate how designers initiate and continually engage with a generative AI system to achieve consistency of outputs and elaborate design variations. The literature review identifies variables to assess the impact of creative input to structure the generative AI visualizations. Our analysis focus on the cognitive workflows and its ability to integrate a mix of analogical and digital methods( sketching, prompts writing, and digital imagery). The analysis constructs focus on the way the design methods translate different spatial qualities, phenomenal experiences that have potential to facilitate social communication and relations, (proximity, sightline-acoustic comfort). Our insights constructs a design framework to guide prompt composition that inform iterative generative AI outputs. The schematic framework may be used to guide AI generated visualizations according to the qualities of atmosphere linked to the spatial forms and arrangements of interior spaces. Written prompts and imagery are examined in their ability to articulate design principle, spatial qualities, spatial characteristics, spatial forms and social relations.
Elaborating a Design Framework That Is Able to Structure and Guide Composition 287
INTRODUCTION
One aspect of using generative AI in design workflows that is not well understood, is how designers navigate the use of prompts and imagery to control design outputs. To learn more about the ways generative AI can elaborate different innovative design aspects linked to a conceptual genre, we conduct an analysis based on designers’ outputs from a recent spec- ulative design workshop and link these to a mapping of iterative design workflow. The analysis of prompts and imagery used to inform generative AI visualizations is an important aspect of this study. Together the insights gained from this study may be used to structure and guide similar undertak- ings and composition of AI visualizations that take advantage of different and innovative design thinking strategies. The discussion further considers insights on how designers were able to achieve satisfaction in their abil- ity to translate their ideas with AI in ways that align to their conceptual vision.
METHDOLOGY This study seeks to answer the following research question: how do design- ers initiate and continually engage with a generative AI system to achieve consistency of outputs and elaborate design variations? Our analysis begins with a description of the speculative design framework. The analysis is structured by a cognitive design framework to characterize inputs and outputs according to spatial structure and spatial qualities, two aspects of generic abstraction (Middleton, 2024). The analysis focuses on three aspects;
1. The way prompt language is used to inform the AI structure of the visualization. 2. The analysis identifies two aspects seen in the visualization outputs that demonstrate the relationship of spatial structure and spatial quality in the AI constructed visualizations. 3. Visualization quality is then assessed with consideration of fidelity and consistency across the iterative visualization sequence.
Figure 1: Framework analysis.
SPECULATIVE DESIGN FRAMEWORK A workshop “The Quality that Lights Up”, was held on a Chinese uni- versity to investigate possible futures of lighting, it was structured as two phases: phase I Vision and phase II Solution. In phase I designers collab- orate in teams and access the virtual platform, Miro to share ideas and strategic foresight about the future of lighting (Fig. 3). This speculative design process follows a methodological approach (Balagtas, 2019, Candy, 2018) to research drivers, trends and identifying focal issues, (Santer, 2019) and construct futuristic scenarios archetypes. (Fergnani, 2019) The sce- nario architypes were used to structure further analyses and shape probable spatial outcomes with lighting experiences. The initial workshop structure (Fig. 2) was organized into two phases of five days each, with the first phase focusing on defining a vision and the second phase articulating the materialization of solutions. A probability impact matrix was used to orga- nize the analysis of trends research and design priority guiding designers to engage in the planning of scenario architypes and to frame a design future within a specific time horizon of ten years. Designers then use analogical and digital methods: hand sketches, post-it pin-up, storyboard, physical space configurations to translate their design intent and provide input into the AI tools.
Figure 2: Workshop workflow across two phases (ten days).
Elaborating a Design Framework That Is Able to Structure and Guide Composition 289
Figure 3: Miro speculative design framework phase I vision.
ANALYSIS AND FINDING Two examples of designers informed iterative processes are shown below. The young designers were asked to visualise their idea through different AI tools. This is shown below as inputs of prompt and sketch and outputs of AI visualizations.
Table 1. Workshop outcome sample designer 1 phase I using AI Vizcom.
Elaborating a Design Framework That Is Able to Structure and Guide Composition 291
Table 2. Workshop outcome sample designer 1 phase II using Midjourney.Table 3. Workshop outcome sample designer 2 phase I using Midjourney.
Elaborating a Design Framework That Is Able to Structure and Guide Composition 293
Table 4. Workshop outcome sample designer 2 phase II using Midjourney.
CONCLUSION
One of the outcomes requested to the designers was to maintain the control over their idea without falling into the easy fascinations given by the images iteration generated from the different AI tools (Vizcom and Midjourney), focusing on obtaining consistency between their research conclusion, vision and AI outputs. Our analysis first began by contrasting the input and output between Vizcom and Midjourney.
In the first sample Tab. 1, following the speculative design analysis, the designer started his investigation with Vizcom (workshop phase I) utilizing sketches, as analogical source, and prompts to instruct the AI tool; the result were quickly iterated, (one day, without any previous knowledge of the AI tool), and suitable results were produced for final selection. In the Tab. 2 (workshop phase II), the designer utilized Midjourney for his outcomes. In his investigation process, prompts and the images iterations were the main inputs to instruct the AI; the designer utilized the AI tool as a smart assistant to engage and develop his design idea. The four different iterations shows how thanks to the proposed framework and workflow the young designer was able to scale up his idea from a smart furniture to an integrate design system.
In the second sample Tab. 3 and Tab. 4, the designer preferred to use Mid- journey in both workshop phases, the investigation conducted during the speculative design analysis helped to instruct the AI tool with clear infor- mation. The designer had already some basic knowledge of the AI tool and this is reflected in the advanced prompts composition. During the workshop phase I, the outcomes are diversified in four different scenarios, the AI tool was used to quickly translate the different investigations into design visual- izations. During the workshop phase II, the designer were instruct to focus on one of this futuristic scenario and developed it. As show in Tab. 4 the iter- ations and prompts composition reflect the different stages of the creative process, the AI tool were used to explore different feasible design outcomes.
The second form of analysis is structured by the cognitive design frame- work to characterize inputs and outputs according to spatial structure and spatial qualities, two aspects of generic abstraction. The way this spatial con- structs were formed in prompts and sketches to instruct AI tools to translate them into visual inspiration to achieve tangible and consistent design results. The aim of this research is to provide an insight into how, using a structured framework, it is possible to use design conception and enable designers to instruct and inform AI tools. The analysis demonstrates how designers were capable to maintain control of AI visualization and satisfy their design intent. The main challenges of this workshop for the facilitators was to create a framework able to combine the proposed speculative design charette in com- bination with hyper-interactive spaces (Albano et al., 2024), and find suitable AI visualization platforms that would work with designer inputs, sketches and prompts (analog and digital), to generate AI outputs consistent with their design intent. One of the findings is that more proficient designers became in composing the prompts commands the better the design outcomes became. The rapid learning curve demonstrated by young designers to use and refine
Elaborating a Design Framework That Is Able to Structure and Guide Composition 295
AI images through this constructed framework, led to increase the iterative design cycles and generated excitement to structure prompts syntax. This process was inspirational and capable to translate their idea, starting from generic abstraction, into design results with great consistency and immediacy. We can conclude that any design process produces more effective outcomes when the instructions are clear; thus, through a combination of a structured framework and a skilled use of AI tools designers were able to achieve a sat- isfactory level of consistency, increase their productivity, and explore more creative and aesthetic aspects.
REFERENCES
Albano Silvia; Middeton Deborah; Longo GianMarco. 2024. How can physical studio space integrate and support speculative AI design experimentation and visu- alization workflows? (Forthcoming) IHIET-AI, AHFE International Proceedings 2024. Balagtas Phil 2019, Design Is [Speculative] Futures Design Thinking - a new
toolkit for pre-emptive design. https://www.youtube.com/watch?v=UB9UVHGI 6AI Accessed (6 November 2023) Candy, S. 2018. Gaming futures literacy: The Thing From The Future. In: MILLER, R. (ed.) Transforming the Future: Anticipation in the 21st Cen- tury. United Nations Educational, Scientific and Cultural Organisation. Enjellina, E. V. P. Beyan, and A. G. C. Rossy, ‘A Review of AI Image Generator:
Influences, Challenges, and Future Prospects for Architectural Field’, Journal of Artificial Intelligence in Architecture, vol. 2, no. 1, Art. no. 1, Feb. 2023, doi: 10.24002/jarina.v2i1.6662 Fergnani A, Jackson M. Extracting scenario archetypes: A quantitative text analysis of documents about the future. Futures&Foresight Science. 2019;1: e17. https://doi.org/10.1002/ffo2.17 IES Visionary Challenge, 2020 https://design.umn.edu/sites/design.umn.edu/file s/2021-09/mary-guzowski-profile-publications-visionary-challenge-2020.pdf Accessed (6 November 2023). Microsoft, 2023. https://news.microsoft.com/source/features/ai/the-art-of-the-pro
mpt-how-to-get-the-best-out-of-generative-ai/ Accessed (6 November 2023). Middleton, Deborah, 2024. Evaluating Human-AI Design Interaction from the perspective of Cognitive Abstraction. (Forthcoming) IHIET-AI, AHFE International Proceedings 2024. Steven Santer 2019, Drivers, Trends, and Mega-Trends. https://futurehumanbydesig
n.com/2019/10/drivers-trends-and-mega-trends/ Accessed (4 November 2023).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-96-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional