Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Human-Centered Decision Support for Data Analytics in Production Systems

Technische Universität Berlin, Institute of Machine Tools and Factory Management, Department of Quality Science, Pascalstr. 8-9, 10587 Berlin, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1007780

Abstract

Manufacturing companies increasingly rely on production data to enable data-driven decisions, yet the feasibility and reliability of analytics are often constrained by insufficient data quality. To support practitioners in selecting suitable analytics methods under real-world constraints, this paper validates a human-centered decision-support approach that links a data-quality maturity assessment to a catalog of production- and quality-related analytics methods. Building on an ISO/IEC based process assessment logic and data-quality measures, the approach was evaluated through an industrial single-case study in a manufacturing plant. An exploratory analysis of an SAP quality dataset informed three semi-structured expert interviews spanning return-quality analytics, production-quality analytics, and data engineering. The scoped assessment focused on the data lifecycle phase Data Processing and selected measures for completeness, consistency, and currentness. Findings show that completeness and semantic accuracy are the most consequential limiting factors for downstream analytics, while duplicates and some inconsistencies can be mitigated effectively through automated, rule-based controls. Data quality is shown to be a socio-technical outcome shaped by enterprise systems, process design, and work practices. Documented processes may diverge from lived practice, and automation can both reduce input variability and introduce new failure dependencies. Based on the case evidence, the paper derives practical requirements for maturity-based decision support, including reduced implicit expertise demands, clearer separation of process and data documentation versus execution, and explicit checks for operational process adherence and automation context.

INTRODUCTION

Manufacturing companies increasingly aim to leverage the availability of production data to improve productivity, reduce downtime, and support continuous improvement (Tao et al., 2018). Smart manufacturing has led to an increase in data, which serves as a basis for data-driven decision-making processes (Bousdekis et al., 2021). However, processing high-volume, high- velocity, and high-variety data remains difficult (Gao et al., 2020).

A main obstacle for industrial analytics is data quality (Müller-Stein et al., 2025). It creates direct and indirect costs and can undermine trust in Received February 20, 2026; Revised April 1, 2026; Accepted April 16, 2026; Available online July 20, 2026

analytics-driven recommendations (Haug, Zachariassen and van Liempd, 2011). Standards such as ISO 8000-8 emphasize data quality concepts and prerequisites for measuring information and data quality in the context of quality management processes (ISO, 2015). In production environments, data quality substantially influences which analytics methods are feasible and what level of confidence can be justified (Gudivada, Apon and Ding, 2017).

Despite its high importance, systematic methods for evaluating and improving data quality are often lacking in practice. Established standards such as ISO/IEC 25012 and 25024 were primarily developed for traditional IT systems (ISO and IEC, 2008, 2015a). Modern smart manufacturing applications, including big data and machine learning applications, are particularly sensitive data quality (Bousdekis et al., 2021). Recent standards such as ISO/IEC 5259-2 propose data quality measures for analytics and machine learning (ISO and IEC, 2024). Maturity models offer an approach to systematically assess and gradually improve data quality by enabling organizations to record their current status and define targeted development paths (Müller-Stein, Vogt and Jochem, 2025).

However, many of the theoretical models lack validation in industrial contexts. The question of whether and how these models are practicable in the reality of manufacturing companies remains largely unanswered. Production and quality management decisions are made by practitioners, and from a human-centered perspective, decision support must be usable, transparent, and aligned with user goals and constraints (ISO, 2019). Therefore, structured, practitioner-oriented guidance that links the current state of data quality and analytics readiness to the selection of methods that can be applied effectively in production systems is needed.

This paper addresses that gap by validating a human-centered decision support approach for method selection in production data analytics. Building on a data quality maturity model (Müller-Stein, Vogt and Jochem, 2025), an assessment is combined with a method catalog relevant to production and quality contexts (Müller-Stein, Li and Jochem, in press). The validation is conducted via a case study in an industrial context (Yin, 2018).

The remainder of the paper presents the concept and method, reports the industrial validation and results, and concludes with implications for practitioners, limitations, and future work.

CONCEPT AND METHOD In smart manufacturing, interconnected production resources, sensors, and information systems continuously generate data that is exploited to improve performance, quality, and maintenance through analytics (Mourtzis, 2024). The reliability of analytical insights is constrained by the quality of the underlying data, which becomes more critical as organizations move from descriptive reporting towards predictive and prescriptive approaches that rely on statistics and machine learning (Sharma et al., 2022).

To address data quality systematically, standards provide a structured model and measurable indicators. ISO 8000-2 defines data quality as the “degree to which a set of inherent characteristics of data fulfils requirements”

Human-Centered Decision Support for Data Analytics in Production Systems 99 (ISO, 2022). ISO/IEC 25012 operationalizes this by specifying data quality characteristics and distinguishing between inherent and system-dependent characteristics (ISO and IEC, 2008). ISO/IEC 25024 complements the model with defined quality measures linked to phases of the data lifecycle (ISO and IEC, 2015a). For analytics and machine learning, ISO/IEC 5259-2 introduces additional characteristics and measures, such as representativeness and balance, that can directly affect model validity (ISO and IEC, 2024).

In order to build capability over time, companies use maturity models to assess current practices and guide staged improvement (Becker, Knackstedt and Pöppelbuß, 2009). A data-quality maturity model for predictive analytics in manufacturing is used (Müller-Stein et al., 2025; Müller-Stein, Vogt and Jochem, 2025) following the process assessment logic of ISO/IEC 33000-2 (ISO and IEC, 2015b). The assessment is based on capability levels, with explicit criteria for each category. Figure 1 shows the flow chart for its practical application.

Figure 1: Flow chart showing the practical application of the maturity model. Based on ISO and IEC (2015b).
Original PDF Figure 1:

The maturity model was validated using a single-case study design according to Yin (2018), which includes a qualitative content analysis of expert interviews in a manufacturing plant. In preparation for the interviews, an exploratory data analysis was conducted on an SAP quality-related dataset to identify specific data quality issues and to create the interview guide. The maturity model was applied in a scoped form. The assessment concentrated on the data lifecycle phase Data Processing. The quality assessment was further reduced to the following measures shown in Table 1.

Table 1: Selected data quality measures from ISO and IEC (2015a).

Table available in the original PDF paper.

ID Dimension Measure Description Com-I-2 Completeness Attribute completeness

Completeness of data items within a data file

Con-I-3 Consistency Risk of data inconsistency

Risk of having inconsistency due to duplication of data value Con-I-6 Consistency Semantic consistency Degree to which semantic rules are respected

Cur-I-1 Currentness Update frequency Degree to which data items are updated with the frequency required After preliminary discussions with experts, the dimension of accuracy was omitted, as semantic accuracy would only be possible to verify with considerable effort and syntactic accuracy is negligible for entries that are partially generated by machines. Furthermore, dimensions such as credibility, accessibility, compliance, and confidentiality were not included, as these are already guaranteed by the system architecture and are not applicable to internally generated production data without regulatory requirements.

Empirical evidence was collected via three semi-structured interviews with experts selected for functional expertise and complementary roles along the data value chain (Kaiser, 2021). The fields return-quality analytics, production-quality analytics (dashboard/ad-hoc analysis), and data engineering were covered. Interviews were conducted online for 60 minutes each. The interview guide covered role/context, observed data-quality issues, process and organizational/technical conditions, and validation of the maturity model and its assessment questions.

Transcripts were analyzed using qualitative content analysis with inductive category development, enabling both model-related evidence and practice- driven themes to emerge (Mayring and Fenzl, 2019). Results were synthesized into themes that informed the maturity assessment and practical implications for improving data quality.

CASE STUDY FINDINGS The statements made by the experts were divided into four main categories: data quality, processes, technical systems, and maturity model.

Data Quality Across all interviews, completeness is identified as a core problem. For returns-related data, missing entries in the defect-description field often become visible only during specific analyses, and it is estimated that roughly 5 % of fields are left unfilled incorrectly. Moreover, records created without a corresponding business transaction severely reduce traceability and analytical usability. In production-quality data, issues with completeness are confirmed but less pronounced, which is attributed primarily to higher automation and machine-populated fields.

Semantic accuracy was also reported to be critical. Employees may interpret defect categories differently, while detailed justifications are relegated to

Human-Centered Decision Support for Data Analytics in Production Systems 101 free-text fields, leading to high variability in data quality. In production- quality data, again, fewer semantic issues due to SAP masks with predefined selections and automated data generation were reported.

Further problems include syntactic errors, which can break links between datasets if the system accepts inputs without adequate validation. In contrast, semantic inconsistencies in timestamps are described as rare due to predominantly machine-generated dates. Duplicates are not considered a major issue, as automated watchdogs were already implemented. The currently reactive nature of error discovery is criticized, as issues are often only noticed when performing analyses or responding to customer inquiries, indicating a lack of systematic, automated quality control during data entry and an overreliance on individual skills.

Processes The interviews showed that process design and execution are the primary levers for improving data quality, often outweighing technical interventions. It was emphasized that good processes are the basis for good data quality, and that without plausibility checks and appropriately defined mandatory fields, data quality will inevitably deteriorate.

At the same time, the interviews emphasize that processes do not address all data-quality dimensions equally. A distinction should be made between dimensions that can be standardized and enforced versus those that remain highly user-dependent. Processes are portrayed as particularly effective for completeness, whereas accuracy depends more strongly on individual judgment during data entry.

A central organizational factor is the dependency on the qualifications of individual employees. A substantial variation in skill levels is reported, while briefly trained temporary workers may be expected to perform the same documentation tasks as experienced employees. As a mitigation, an additional, specially trained person conducting systematic plausibility checks is proposed. However, this remains a trade-off between quality and cost.

Technical Systems The interviews show that technical measures can mitigate data-quality issues, while also introducing new dependencies and failure modes. A key example is the introduction of a shop-floor data acquisition application, which should have reduced human error through guided input and automated handling of records. It is emphasized that technical automation does not eliminate completeness risks, as there were instances where data were suddenly missing because of issues with the application, underscoring that data quality becomes contingent on system availability and reliable interfaces rather than user behavior.

In addition to front-end process support, implementing automated data- quality monitoring should be done via watchdogs that execute scripts and check data quality in the background. This could mitigate multiple classes of errors. A duplicate check for redundant files created by parallel processing and content-based validations for business-rule violations and missing required

linkages are possible applications. In practice, these watchdogs have proven effective for simple, repeatable checks, and data-quality metrics were applied implicitly by translating operational experience into automated rules.

Finally, structural limitations of downstream quality checks are highlighted. Error detection for data that must be extracted comes too late, because entries are already in the system and require costly retrospective correction. This reinforces the need to balance timely prevention, automated detection, and the organizational capacity to fix identified issues.

Maturity Model The maturity model itself was also analyzed. It was concluded that its overall logic is understandable, but its practical applicability is constrained by structural and conceptual issues. All three experts consider the basic principle plausible and therefore regard the model’s structure as fundamentally sensible, with the following limitations.

A key critique concerns the implicit prerequisites for using the model. Even for a simple completeness check, the model must clarify when data are complete, and that answering the questions meaningfully requires domain knowledge and familiarity with ISO-based concepts. It was also questioned whether metric-specific questions are needed or whether more generic questions would be sufficient.

From an operational perspective, the questionnaire’s scope should be discussed. The question set is perceived as too extensive for practical application. Additionally, an explicit question addressing whether a documented process is successfully applied and lived is proposed. Also, having a process is necessary but not sufficient for good data quality.

Despite these limitations, the model provides a systematic framework and can serve as a basis for sensitizing stakeholders to issues such as process implementation, documentation, communication to relevant personnel, and monitoring, while also enabling comparability across organizational units when applied consistently.

CONCLUSION

This study contributes to human-centered decision support for data analytics in production systems by consolidating empirical insights from an industrial case study into actionable implications for assessing analytics feasibility under real-world data quality constraints. The results show that data quality is not primarily a technical artifact but emerges from a socio-technical interaction between enterprise systems, process design, and work practices. Completeness and semantic accuracy remain the most consequential limiting factors for downstream analytics, while error types such as duplicates can be effectively mitigated through automated, rule-based controls, illustrating where technical interventions are most effective.

The findings further indicate that enterprise platforms can affect data quality in both positive and negative ways: while structured input masks and predefined selection options help reduce variability, limited validation

Human-Centered Decision Support for Data Analytics in Production Systems 103 capabilities and the restricted configurability of mandatory fields still permit inconsistent entries and delay error detection. Although effective processes are highlighted as a neccessary foundation for reliable data, gaps may remain between documented and lived processes. This also shows the human- centered nature of data quality: training, interpretation, compliance, and local adaptations shape whether data are captured in a way that supports analytics, even when formal procedures exist.

Based on these insights, the study derives implications for maturity-based decision support. To be practically usable in production settings, maturity assessments should require little implicit domain expertise and distinguish clearly between process documentation, data documentation, process execution, and measurable data-quality outcomes. Moreover, maturity evaluation should explicitly account for context factors such as the degree of automation in data acquisition and should include a dedicated check for whether processes are correctly applied in daily operations.

Overall, the work supports a perspective in which maturity models do not only score data quality, but also function as decision-support interfaces between human work practices and analytics choices. They should help stakeholders identify which analytical methods are feasible, which quality risks are critical, and where interventions yield the greatest impact. Future work should validate these conclusions across multiple cases and broader datasets, and operationalize the proposed model refinements to enable robust, human-centered guidance for analytics deployment in production systems.

REFERENCES

Becker, J., Knackstedt, R. and Pöppelbuß, J. (2009) ‘Developing Maturity Models for IT Management’, Business & Information Systems Engineering, 1(3), pp. 213–222. doi: 10.1007/s12599-009-0044-5 Bousdekis, A. et al. (2021) ‘A Review of Data-Driven Decision-Making Methods

for Industry 4.0 Maintenance Applications’, Electronics, 10(7). doi: 10.3390/ electronics10070828 Gao, R.X. et al. (2020) ‘Big data analytics for smart factories of the future’, CIRP Annals, 69(2), pp. 668–692. doi: 10.1016/j.cirp.2020.05.002 Gudivada, V., Apon, A. and Ding, J. (2017) ‘Data Quality Considerations for Big

Data and Machine Learning: Going Beyond Data Cleaning and Transformations’, International Journal on Advances in Software, 10(1&2), pp. 1–20. Haug, A., Zachariassen, F. and van Liempd, D. (2011) ‘The costs of poor data quality’, Journal of Industrial Engineering and Management, 4(2), pp. 168–193. doi: 10.3926/jiem.2011.v4n2.p168-193 International Organization for Standardization (2015) ISO 8000-8:2015 Data

quality — Part 8: Information and data quality: Concepts and measuring. International Organization for Standardization (2019) ISO 9241-210:2019 Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems. International Organization for Standardization (2022) ISO 8000-2:2022 Data

quality — Part 2: Vocabulary.

International Organization for Standardization; International Electrotechnical Commission (2008) ISO/IEC 25012:2008 Software engineering — Software product Quality Requirements and Evaluation (SQuaRE) — Data quality model. International Organization for Standardization; International Electrotechnical

Commission (2015a) ISO/IEC 25024:2015 Systems and software engineering — Systems and software quality requirements and evaluation (SQuaRE) — Measurement of data quality. International Organization for Standardization; International Electrotechnical Commission (2015b) ISO/IEC 33002:2015 Information technology — Process assessment — Requirements for performing process assessment. International Organization for Standardization; International Electrotechnical

Commission (2024) ISO/IEC 5259-2:2024 Artificial intelligence — Data quality for analytics and machine learning (ML) — Part 2: Data quality measures. Kaiser, R. (2021) Qualitative expert interviews: Conceptual foundations and practical implementation. 2nd edn. Wiesbaden: Springer VS. Mayring, P. and Fenzl, T. (2019) ‘Qualitative Inhaltsanalyse’, in Baur, N. and Blasius,

J. (eds.) Handbuch Methoden der empirischen Sozialforschung. Wiesbaden: Springer VS, pp. 633–648. Mourtzis, D. (2024) ‘2 - Industry 4.0 and smart manufacturing’, in Mourtzis, D. (ed.)

Manufacturing from Industry 4.0 to Industry 5.0: Elsevier, pp. 13–61. Müller-Stein, L.F. et al. (2025) ‘Smart Assessment of Data Quality for the Application of Data Analytics in Manufacturing SMEs’, IFAC-PapersOnLine, 59(10), pp. 853–858. doi: 10.1016/j.ifacol.2025.09.145 Müller-Stein, L.F., Li, L. and Jochem, R. (in press) ‘Development of a methodology

catalog for the use of data analytics in production’, in Wang, W.-H. and Vaino, V. (eds.) Quality and Artificial Intelligence – Challenges and Opportunities in Medical Technology and Other Industries. Report on the GQW Annual Conference 2025 in Lübeck. Wiesbaden: Springer Vieweg. Müller-Stein, L.F., Vogt, T.-O. and Jochem, R. (2025) ‘A Maturity Model For The Application Of Big Data Analytics In Manufacturing SMEs’, in Herberger, D. and Hübner, M. (eds.) Proceedings of the Conference on Production Systems and Logistics: CPSL 2025. Offenburg: publish-Ing, pp. 674–688. Sharma, A.K. et al. (2022) ‘1. Analytics Techniques: Descriptive Analytics, Predictive

Analytics, and Prescriptive Analytics’, in Jeyanthi, P.M. et al. (eds.) Decision Intelligence Analytics and the Implementation of Strategic Business Management. (EAI/Springer Innovations in Communication and Computing). Cham: Springer International Publishing, pp. 1–14. Tao, F. et al. (2018) ‘Data-driven smart manufacturing’, Journal of Manufacturing Systems, 48, pp. 157–169. doi: 10.1016/j.jmsy.2018.01.006 Yin, R.K. (2018) Case Study Research and Applications: Design and Methods. 6th

edn. Thousand Oaks, CA: SAGE Publications.

Cover of the proceedings volume Industrial and Systems Engineering

Explore the complete digital volume

Industrial and Systems Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-95-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →