Yifan Yang, Hui Li (2025). Design Evaluation System of AI-Generated Content in the Industrial Design of Construction Machinery. In: Human-Computer Interaction & Emerging Technologies. AHFE Open Access, vol. 195, pp. 61–70. https://doi.org/10.54941/ahfe1006224
The application of Artificial Intelligence Generated Content (AIGC) technology in the industrial design of construction machinery has developed rapidly. However, its generated solutions differ significantly from designer-generated solutions, posing new challenges to traditional design evaluation methods. To address the evaluation challenges of AIGC solutions, this study proposes a design evaluation framework that combines comprehensiveness and efficiency. An evaluation framework based on the Analytic Hierarchy Process (AHP) was developed through expert interviews and literature analysis. The framework includes five criteria and fifteen sub-criteria to comprehensively assess the quality of AIGC design solutions. Subsequently, the evaluation indicators were streamlined and optimized to enhance efficiency. Experimental validation using practical construction machinery design cases demonstrated that this framework maintains scientific rigor while improving the efficiency of screening and decision-making for AIGC solutions. This study provides an efficient and reliable method for the preliminary scoring and filtering of AIGC solutions in the industrial design of construction machinery, contributing to improved decision-making processes in industrial design practices.
In recent years, Artificial Intelligence Generated Content (AIGC) technology has achieved remarkable progress in industrial design, particularly in construction machinery design, where its generative capabilities have significantly improved design efficiency. However, AIGC solutions in the early design stages are characterized by their vast quantity and highly variable quality, presenting significant challenges to traditional design evaluation methods, especially in rapid screening and decision-making processes. For instance, traditional multi-criteria evaluation systems excel at assessing a small number of high-quality design solutions but become overly complex and inefficient when applied to the large-scale outputs of AIGC.
To address these challenges, this study proposes a novel design evaluation method. First, through expert interviews and literature analysis, the study developed an evaluation framework based on the Analytic Hierarchy
Process (AHP), comprising five criteria (Product Form, Colour Coating, Ergonomics, Manufacturing Process, and Brand Identity) and fifteen sub- criteria to comprehensively assess the quality of AIGC design solutions. Subsequently, to meet the demands of rapid evaluation, the framework was streamlined into four criteria (Product Form, Colour Coating, Ergonomics, and Manufacturing Process) through technical analysis and expert panel discussions. This study provides an effective solution for evaluating AIGC design solutions in construction machinery industrial design and establishes a theoretical and practical foundation for its broader application in industrial design.
APPLICATION AND DEVELOPMENT OF AHP IN CONSTRUCTION MACHINERY DESIGN EVALUATION The Analytic Hierarchy Process (AHP) is a structured multi-criteria decision- making method that integrates qualitative and quantitative evaluations. Proposed by Thomas L. Saaty, AHP combines quantitative and qualitative analysis to analyse the influencing factors of decision-making problems, determine weights, and establish priority rankings. It has proven to be highly effective in addressing complex decision-making scenarios. This method has been widely applied in the field of engineering machinery design, which often requires balancing aesthetics, functionality, and manufacturability.
In the context of engineering machinery, AHP has been extensively used to evaluate product appearance and functional consistency, laying a foundation for systematic design evaluation. AHP theory has also been employed in evaluation studies based on user perceptions (Yi and Li, 2021). Researchers have further expanded these frameworks by incorporating methods such as fuzzy logic and entropy weighting to improve the accuracy of weight calculations and address the inherent subjectivity of product design evaluation (Chen et al., 2015; Wang et al., 2024). Recent studies have also explored the inclusion of sustainability metrics and usability testing within the AHP framework, demonstrating its adaptability in the evaluation of engineering machinery.
Characteristics of AIGC-Generated Industrial Design Schemes in Engineering Machinery The application of Artificial Intelligence Generated Content (AIGC) technology in the field of design has become increasingly widespread, offering more efficient and creative solutions across multiple industries. In the industrial design of construction machinery, which requires a balance between functional precision and aesthetic integration, AIGC tools have significantly improved the design process. Midjourney provides a quick and effective means for exploring creative concepts, generating numerous design solutions in a short time, making it especially suitable for the early stages of design conceptualization. Meanwhile, WebUI and ComfyUI platforms, powered by large models such as Stable Diffusion and Flux, offer different workflow supports. WebUI focuses on accessibility and simplicity, enabling designers to control the generated results with minimal technical
Design Evaluation System of AI-Generated Content in the Industrial Design 63
expertise. ComfyUI, on the other hand, allows designers to create customized workflows, offering greater flexibility for complex design tasks. Together, these tools support a diverse range of design needs, from rapid conceptual exploration to detailed customization.
Interviews with five designers experienced in AIGC applications in construction machinery were conducted to compare the characteristics of Midjourney, Stable Diffusion (WebUI), and Stable Diffusion (ComfyUI) with human designers. A common feature of AIGC tools is their ability to generate a large number of solutions with advantages in flexibility and efficiency. However, the stability of solution quality is generally inferior to that of human designers. AIGC schemes typically have high quantity but inconsistent quality. While the ability to quickly generate diverse solutions is a major advantage, it requires additional filtering to identify high-quality designs. In contrast, human-generated schemes are fewer in number but exhibit greater consistency due to the direct involvement of professional expertise. This difference highlights the importance of establishing an evaluation framework tailored to the characteristics of AIGC-generated schemes.
In conclusion, AIGC technology provides valuable support for enhancing creativity and shortening the time needed for conceptual exploration in construction machinery design. However, the differences between AIGC-generated and designer-generated schemes suggest the necessity of establishing an evaluation system that balances efficiency and adaptability while ensuring consistency and scientific rigor. This study aims to address these challenges and lay a foundation for the comprehensive assessment of AIGC schemes in industrial design.
Figure 1: Comparison of midjourney, stable diffusion (WebUI), stable diffusion (ComfyUI), and human designers.
64 Yang and Li
Establishment of Evaluation Criteria for AIGC in Construction Machinery Design To establish a comprehensive evaluation system for AIGC-generated schemes in construction machinery design, this study first conducted a literature review and semi-structured interviews with seven experts and designers experienced in both construction machinery design and AIGC applications. Through these interviews, relevant keywords were extracted, screened, and refined. These keywords were subsequently discussed and validated by an expert panel, leading to the development of the evaluation framework presented in the table below.
Table 1: Evaluation criteria and sub-criteria for AIGC in construction machinery design.
Table available in the original PDF paper.
Criteria Sub-Criteria Description
1. Product Form (B1) Overall Sense (C1) Reflects the coherence and visual
impact of the overall product form. Proportional Form
Ensures that the proportions of the
Equations and mathematical formulas are available in the original paper PDF.
product are visually pleasing and functionally appropriate. Distinct Product
Highlights unique design features
Features (C3)
that align with the identity and purpose of the machinery. Form and Function
Ensures that the design integrates
Alignment (C4)
functionality seamlessly with aesthetic appeal. 2. Color Coating (B2) Color Harmony (C5) Assesses the compatibility and
Evaluates how well the color and
balance of colors used in the design. Color and Material Function Alignment (C6)
Examines whether the colors and
material choices support the functional requirements of the product. Semantic Accuracy of Color and Material (C7) materials accurately convey the intended design semantics or brand image. Aesthetic Appeal of
Measures the visual attractiveness
Coating (C8)
and finish quality of the product’s coating. 3. Ergonomics (B3) Reasonable
Ensures that the dimensions of
Component Dimensions (C9)
components are appropriate for intended use and user interaction. Functional Layout
Evaluates the logical arrangement
and accessibility of functional elements. Reasonable
Assesses whether the structural
Structural Design (C11)
design aligns with usability and operational efficiency.
Continued
Design Evaluation System of AI-Generated Content in the Industrial Design 65
Table 1: Continued Criteria Sub-Criteria Description
Table available in the original PDF paper.
Reasonable
Evaluates the cost-effectiveness of
4. Manufacturing Process (B4)
Production Costs (C12)
the production process without compromising quality. Reasonable
Assesses whether the design is
Manufacturing Feasibility (C13)
practical and feasible to produce using available manufacturing techniques. 5. Brand Identity (B5) Brand Continuity
Measures the alignment of the
design with the brand’s established identity and characteristics. High Brand
Evaluates the clarity and
Recognition (C15)
effectiveness of the design in enhancing the product’s recognizability and brand association.
Calculation of AHP Criteria Weights After determining the criteria and sub-criteria in the previous step, the next phase involves constructing pairwise comparison matrices. To achieve this, a panel of seven experts and designers with extensive experience in both construction machinery design and AIGC applications was assembled to conduct the pairwise comparisons. The experts evaluated the relative importance of each element within the criteria hierarchy, using the standard AHP 1–9 scale to quantify their judgments. Constructing the judgment matrix A, where aij represents the degree of importance of element i compared to j.
Table 2: Nine-point scale table for AHP.
Table available in the original PDF paper.
Scale Definition Explanation
1 Equally Important Two elements contribute equally to the property
One element is slightly more important than the
being compared. 3 Moderately More Important
One element is significantly more important than
other with respect to the property. 5 Strongly More Important One element is very strongly more important than the other with respect to the property. 7 Very Strongly
More Important One element is absolutely more important than
the other with respect to the property. 9 Extremely More Important Used when a compromise between two adjacent the other with respect to the property. 2, 4, 6, 8 Intermediate
Values
judgments is necessary. Reciprocal Reciprocal Values If the importance of element i over element j is aij,
66 Yang and Li
Calculate the geometric mean of each row.
n Y
Normalize the weights.
i Pn
Calculate Consistency: To verify consistency in the judgments, calculate the following: Calculate the maximum eigenvalue (λmax):
(Aw)i
n X
Among them, A is the judgment matrix, and w is the weight vector; (Aw)i represents the i-th element of the product of matrix A and vector w.
n −1 Consistency Ratio (CR):
Here, RI (Random Index) depends on the size of the matrix.
Judgment Consistency: If CR < 0.1, the pairwise comparisons are consistent and acceptable. Otherwise, the judgments need to be revised.
Evaluation System Construction and Post-Processing Based on the statistical analysis and organization of expert scores, this study constructed the following matrix table to represent the relative importance relationships between criteria and sub-criteria. Each element in the judgment matrix reflects the pairwise comparison result for the corresponding criteria, with its value indicating the degree of relative importance based on AHP’s nine-point scale.
Design Evaluation System of AI-Generated Content in the Industrial Design 67
Table 4: Judgment matrix.
Table available in the original PDF paper.
Criteria Product Form Color Coating Ergonomics Manufacturing Brand Identity Process
Product Form 1 3 5 4 5 Color Coating 1/3 1 3 2 4 Ergonomics 1/5 1/3 1 2 3 Manufacturing Process Brand Identity 1/5 1/4 1/3 1/3 1
The following data can be obtained from the above calculation process: w = [0.485, 0.235, 0.129, 0.108, 0.042]
Consistency Ratio CR<0.1, indicating that the judgment matrix passes the consistency check.
Through calculations and systematic organization of the weights for criteria and sub-criteria, this study ultimately derived the following weight table. The weight allocation is based on the pairwise comparison matrices constructed using the Analytic Hierarchy Process (AHP) and is informed by expert scoring and evaluation. Initially, pairwise comparisons of the criteria layer were conducted to calculate the relative weights of each criterion. Subsequently, the weights of sub-criteria were distributed according to the calculated criteria weights. To ensure the scientific validity and consistency of the results, the consistency of the judgment matrices was verified, and the validity of the comparisons was confirmed through the Consistency Ratio (CR).
Table 5: Table of criteria and sub-criteria weights.
Table available in the original PDF paper.
Criteria Criteria Weight Sub-Criteria Sub-Criteria Weight Product Form 0.485 Overall Sense 0.1859 Proportional Form 0.1678 Distinct Product Features 0.0576 Form and Function Alignment 0.0737 Color Coating 0.235 Color Harmony 0.0957 Color and Material Function
0.0552
Alignment Semantic Accuracy of Color
0.0451
and Material
Aesthetic Appeal of Coating 0.039 Ergonomics 0.129 Reasonable Component
0.0737
Dimensions
Continued
68 Yang and Li
Table 5: Continued Criteria Criteria Weight Sub-Criteria Sub-Criteria
0.108 Reasonable Production Costs 0.0356 Process Reasonable Manufacturing
0.0724
Feasibility
Brand Identity 0.042 Brand Continuity 0.021 High Brand Recognition 0.021 The results indicate that Product Form is the most critical evaluation dimension, with a weight of 0.485, underscoring the central role of overall form and visual appeal in construction machinery design. This is essential not only for traditional design but also for AIGC-generated schemes, where superior form can significantly enhance a design’s competitiveness. The second most important criterion is Color Coating, with a weight of 0.235, highlighting the importance of color harmony, semantic accuracy, and aesthetic quality in coating, all of which directly influence the visual performance of the design.
In contrast, Ergonomics and Manufacturing Process have relatively lower weights, at 0.129 and 0.108, respectively. This is because shortcomings in these areas during the early stages of AIGC design can be addressed through manual corrections by designers or improved control using tools like ControlNet. However, overly impractical ergonomic or manufacturing process designs are still unacceptable, so these dimensions, though less emphasized, must maintain a baseline level of feasibility.
As for Brand Identity, it holds the lowest weight of 0.042. This reflects the minimal variation in brand identity performance among AIGC-generated schemes when using the same, well-suited LoRA (Low-Rank Adaptation) models. Particularly in the field of construction machinery, specialized LoRA models tailored for specific brands can achieve consistent brand identity representation, sometimes even replicating unique brand characteristics. Consequently, Brand Identity is assigned a lower priority in the evaluation system, though it retains some importance in specific scenarios.
Rapid Screening and Simplified Evaluation System for Large-Scale AIGC Design Schemes In the early stages of design tasks, AI tools often generate dozens of AIGC design schemes that meet the required evaluation standards. Although the comprehensive evaluation system described above is scientifically valid and effective, it can consume significant time and resources in practice. After discussions among the expert panel, it was concluded that for the initial screening phase, the evaluation of a large number of schemes can be efficiently conducted by scoring only at the criteria level. This approach significantly reduces the time required for evaluation. Furthermore, with the proper application of technologies such as LoRA and ControlNet, differences in
Design Evaluation System of AI-Generated Content in the Industrial Design 69
Brand Identity among the schemes can be considered negligible. Accordingly, this study proposes a simplified evaluation system designed to meet the needs of preliminary screening for a large volume of design schemes, offering an efficient evaluation tool.
Table 6: Rapid evaluation weight table.
Table available in the original PDF paper.
Criteria Product Form Color Coating Ergonomics Manufacturing Process Weight 0.548 0.234 0.119 0.098
The simplified evaluation system bridges the gap between traditional evaluation frameworks and the demands of large-scale AIGC solutions, ensuring efficient decision-making while maintaining acceptable levels of accuracy. Following the framework simplification, the simplified evaluation system was applied in practical construction machinery design projects to validate its screening efficiency and effectiveness. The results demonstrated that the system could rapidly identify high-quality schemes during the preliminary screening phase, with its accuracy and efficiency acknowledged by the expert team.
CONCLUSION
This study developed a comprehensive evaluation framework for AIGC- generated construction machinery design schemes, integrating expert knowledge with the AHP methodology. By establishing criteria and sub- criteria weights, the framework provides a structured and scientific approach to evaluate the aesthetic, functional, and manufacturing aspects of AIGC schemes. Furthermore, to address the practical challenges of evaluating a large number of initial design schemes, a simplified evaluation system was proposed, allowing for rapid and efficient preliminary screening. This research not only supports the practical application of AIGC in industrial design but also lays a theoretical foundation for future advancements in automated design evaluation systems.
REFERENCES
Chen, Minna, et al. (2015). Evaluation of Coal Mine Machinery Industrial Design Based on Fuzzy AHP, Journal of Taiyuan University of Science and Technology, 36(03), pp. 223–227. Huang, K. L., Liu, Y. C., Dong, M. Q., and Lu, C. C. (2024). Integrating AIGC into
product design ideation teaching: An empirical study on self-efficacy and learning outcomes, Learning and Instruction, 92, 101929. Ji, Xianghong, and Xia, Yuzi. (2024). Innovative Design of Hunan Local Product Packaging under the Mode of “Intangible Cultural Heritage + AI”, Packaging Engineering, 45(16), 301–312. Li, Xuenan, Zhao, Jianghong. (2013). Study on Consistency of Form Feature and
Semantics of Construction Machinery, Packaging Engineering, 34(02), 61–64. Li, Zehong, et al. (2023). Evaluation of Intelligent Hydraulic Equipment Design Based on AHP-TOPSIS Method, Packaging Engineering, 44(14), 83–90.
70 Yang and Li
Lin, H., Jiang, X., Deng, X., Bian, Z., Fang, C., and Zhu, Y. (2024). Comparing AIGC and traditional idea generation methods: Evaluating their impact on creativity in the product design ideation phase, Thinking Skills and Creativity, 54, 101649. Qi, Yuzhe, and Kiesu Kim. (2024). Evaluation of electric car styling based on analytic
hierarchy process and Kansei engineering: A study on mainstream Chinese electric car brands, Heliyon, 10(5). Wang, Weixu, et al. (2024). Evaluation on Industrial Design Scheme of Iron Roughneck Based on TFN-AHP Entropy Weight Method, China Petroleum Machinery, 52(07), pp. 70–77. Yan, Yuting. (2015). Research on Modeling Design of Engineering Machine Based
on Semantics, Chang’an University, MA thesis. Yang, Jingling, and Chen, Yanwen. (2024). Innovative Application of AIGC-based Taohuawu Woodcut New Year Paintings in Home Furnishing Design, Packaging Engineering, 45(12), 465–473. Yi, Jun, and Li, Xue. (2021). Engineering Machinery Modeling Evaluation Based on
User Perception, Packaging Engineering, 42(24), 161–168. Yin, Yukun, Chen, Hong, and Zhao, Haiying. (2020). The Application of Artificial Intelligence in Art Design, Packaging Engineering, (06), 252–261.
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-71-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional