Zhang Yao, Zhao Yating, Yihang Du (2022). Construction of Comprehensive Evaluation Model for Non-Technical Skills of Art Teacher under Cross-Discipline. In: Training, Education, and Learning Sciences. AHFE Open Access, vol. 59, pp. 172–179. https://doi.org/10.54941/ahfe1002398
Non-technical skills (NTS) refer to the cognitive and interpersonal capability contributing to working performance according to occupation requirements. The NTS of arts education teachers improves education and teaching quality in the art discipline. To evaluate arts education teachers' NTS, we first designed a questionnaire analyzing four dimensions: decision-making, communication skills, self-management, and situational awareness. We extracted twelve indexes of NTS through the data analysis and established an evaluation model. Subsequently, we adopted the analytic hierarchy process (AHP) for weight calculation and constructed a comprehensive evaluation model of art teachers. This model can provide a practical method and basis for evaluating, training, and selecting art teachers in colleges and universities and improving their NTS.
The development of disciplines promotes the evolution of society, which in turn promotes the adjustment of disciplines (Peng, 2021). Karl Popper said, “We are not students of some subject matter, but students of problems. And problems may cut right across the borders of any subject matter or disci- pline “ (Steph and Machiel, 2016). As problems become more extensive and complex, traditional art research methods, which often relies on a sin- gle discipline or specialty, can no longer cope with the latest changes when answering questions, solving challenges, and dealing with issues. Therefore, besides outstanding professional skills, art teachers must learn how to form a suitable education atmosphere and communicate efficiently with students in daily teaching (as shown in Figure 1), while NTS are the core factor in their competency.
NTS, also known as soft skills, gained attention in education as early as 1967 when the U.S. government adopted the Competency-Based Teacher
Construction of Comprehensive Evaluation Model for Non-Technical Skills 173
Education (CBTE) in the Teacher Education Reform, in which NTS emph- asized the qualities that teachers should possess in addition to professional skills. McClelland proposed the competency model in 1973 as a compreh- ensive representation of technical and NTS, and since the introduction of the concept of competency, many theoretical and empirical results have been obtained from related studies (McCelland, 1973). Bisschoff and Grobler stu- died teachers’ competency using a structured questionnaire and proposed a two-dimensional model of teacher competency and collaborative compete- ncy, including NTS such as leadership, teamwork, and communication skills (Bisschoff and Grobler, 1998). Dineke (2004) proposed a model of com- petency qualities of university teachers in different teaching and research scenarios, where personality traits, attitudes and values are essential aspe- cts of NTS (Dineke et al., 2004). Christian Seufert (2022), in the competent qualities of teachers in virtual reality classrooms, found that instructional management classroom management (CM) is the most important NTS for teachers through quantitative assessment (Christian S et al., 2022). Hong-Yan used the Modified Delphi method to extract NTS such as Selflessness, Com- pressive ability, Ability to adapt, and Fairness Justice for teacher education using nursing faculty from eight universities (Hong-Yan et al., 2021).
The existing studies provide a basis for constructing a model of NTS for art teachers. However, art education has exceptional characteristics, and the critical factors in the model still need to be further explored. This study first used literature analysis to extract a library of NTS indicators, then designed and implemented a questionnaire with college art teachers as the target audi- ence, and then used hierarchical analysis to determine the weights of each indicator to establish a model of NTS for art teachers. The research results help to provide a more reasonable method and basis for the evaluation, trai- ning, and selection of college art teachers and their own NTS improvement in the context of cross-disciplinary development.
PRELIMINARY CONSTRUCTION OF INDEX SET Based on the literature review in the past five years, we combined the classical literature on NTS and teacher competency model, the initially extracted NTS indicators to art teacher according to professional characteristics under cross- discipline.
Preparation and Implementation of Non-Technical Skills Questionnaire After sorting out 117 works, 35 representative works of literature were selected for in-depth analysis (see Table 1), involving three industries: avi- ation, medical and railroad, from which we extracted five scales for analysis. NOTECH is the most common international behavioral indicator system for NTS (Flin, 2011) and has been popular in aviation, nuclear power plants, and the medical industry. Since many research and practices have proved the system reliable and universal, it can be the basis for NTS indicator set con- struction. ANTS, NOTSS and OTAS are Flin’s modifications and extensions of NOTECH for task-based applications such as anesthesiologists, surgeons,
Table 1. Literatures analysis of NTS.
Table available in the original PDF paper.
Date Scale Field Citation
1998 NOTECHS Aviation, Medical, Marine
(Naonori, 2012) (Kate, 2011)
(Flin, Martin and Goeters K, 2003) (Fletcher et al., 2003) (Philip et al., 2014) (Ho, 2013) (A. Mishra and K. Catchpole, 2008) 2003 ANTS Medical (Fletcher et al., 2003) (Yule et al., 2008) (Flin, 2011) 2005 OTAS Medical (S .Yule and R. Flin, 2005)(Ana, 2014) 2007 NOTSS Medical (Yule et al., 2008) (Flin, 2007) 2011 NTS Railroad, Medical
and trauma care. Therefore, he also includes additional indicators such as stress management and risk perception.
Referring to the crew resource management (CRM) model, the Rail Safety and Standards Board (RSSB) conducted a study on NTS for position require- ment for taxi drivers. They constructed the NTS behavioral indicator system, which described NTS in seven dimensions, adding three items of responsi- bility, workload management and self-management compared to NOTECH. The literature (Flin, Martin and GoetersK, 2003) (Fletcher et al., 2003) (Yule et al., 2008) (Philip et al., 2014) (Kate, 2011) are more significantly represen- tative. They are meaningful inspirations for the construction of NTS models for art teachers, so the indicators of NOTECH, ANTS, NOTSS, OTAS and Rail NTS in literature (Flin, Martin and GoetersK, 2003) (Fletcher et al., 2003) (Yule et al., 2008) (Philip et al., 2014) (Kate, 2011) are used as the basis to initially construct a set of metrics for NTS of metro dispatchers.
We employed the integrated method to extract the indicators of all scales in the literature and de-weighted the indicators with the same name or similar meaning. As shown in Table 2, we removed indicators that were not relevant to art teachers (e.g., flight height perception) and obtained thirty-three indi- cators to construct an initial screening indicator set of NTS for professional art teachers, constructing the initial screening indicator set of NTS for art teachers. As shown in Figure 1, the field survey of art education takes place from September 2020 to July 2021.
Questionnaire Design and Implementation As shown in Figure 2, we established the NTS preliminary model according to the dimensional division of the NOTECH scale.
The questionnaire on NTS of art teachers in colleges and universities analy- zes the four dimensions: communication, teamwork, decision-making, and situation awareness. The survey selected 18 college art teachers who have worked for more than two years and passed the annual teaching assessment involving painting, music, dance, and theatre art. The questionnaire consisted of 33 items, including six for the communication ability, ten for teamwork ability, nine for the decision-making ability, and eight for situation awareness
Construction of Comprehensive Evaluation Model for Non-Technical Skills 175
Figure 1: Arts education field research.Figure 2: Arts teacher NTS model preliminary.
ability, clearly defining the meaning of each skill and setting up the questi- ons. The questionnaire was on a 5-point scale of “not relevant, not very relevant, average, relatively relevant, very relevant”, with values from 1 to 5. We distributed 18 copies of the questionnaire, the return rate was 100%, and the effective rate of the questionnaire was 100%.
Reliability Analysis of the Questionnaire Cronbach’s alpha internal consistency coefficient of this questionnaire was 0.975, higher than 0.9, indicating the high reliability in this study. The value of the Spearman-Brown semi-reliability coefficient is 0.974, which is higher than 0.9, meaning that this research’s data effectively analyses the characteristic pattern of NTS of art teachers in colleges and universities.
CONSTRUCTION OF A COMPREHENSIVE EVALUATION MODEL BASED ON THE AHP METHOD The Initial Selection of Indicators
According to the characteristics of NTS of college art teachers, based on the existing principal components and concerning domestic and foreign references, we used the AHP method to establish a progressive hierarchical structure model, and divided the evaluation indexes into three layers, which are the target layer, criterion layer and sub-criterion layer, see Figure 1. The target layer of the step hierarchy model is the of college art teachers; the crite- rion layer is communication ability B1, teamwork ability B2, decision-making ability B3, situational awareness ability B4. The indicators under B1 in the indicator layer are language application ability C1, logical thinking ability C2, humanistic care ability C3, and under B2 are organizational assistance ability C4, information integration ability C5, psychological regulation abi- lity C6. The indicators under B3 are C7 insightful perception, C8 rational thinking, and C9 problem-solving. The indicators under B4 are C10 infor- mation gathering ability, C11 cognitive judgment ability, and C12 prediction ability.
Questionnaire Design and Implementation The second questionnaire analyzed the NTS model for college art teachers with more than nine years of working experience. The questionnaire has three levels, 12 dimensions, 18 indicators and separate questions for each indicator. The 1-9 assignment method evaluated the significance of indica- tors at each level, with 1 representing “extraordinarily unimportant” and 9 representing “extraordinarily important”. In the second round, we sent out ten questionnaires and recovered ten with a recovery rate of 100%.
Weights Calculation For the framework of indicators that have been screened, a complete recursive hierarchy was formed and using the above, experts in the field were first selected and divided on the critical degree of indicators and then discussed and summarized to obtain a two-by-two judgment matrix and calculate the weights.
Establishing the Judgment Matrix A hierarchy was formed, and using the above, experts in the field were first selected and divided on the critical degree of indicators and then discussed and summarized to obtain a two-by-two judgment matrix and calculate the weights.
Matrix A = (aij)n×n represents the relative importance between n indica- tors, where aij is the experts’ assessment of the importance of factor i relative to factor j
Construction of Comprehensive Evaluation Model for Non-Technical Skills 177
Equations and mathematical formulas are available in the original paper PDF.
(1)
Equation reproduced from the original PDF.
Equations and mathematical formulas are available in the original paper PDF.
The relative weights of the compared elements for the hierarchy elements are calculated from the judgment matrix. Denote the maximum eigenroot as λmax and its corresponding eigenvector using Eq. (2).
(2)
ωi = ωi nP
Equation reproduced from the original PDF.
Consistency Index The judgment matrix has general consistency to calculate the weight vector from the judgment matrix equation. It is necessary to calculate the Consiste- ncy Index (C.I.) = 0.0052.
C.I. = λmax −n
n −1 (3)
Equation reproduced from the original PDF.
R.I. (4) When C.R. < 0.1, the consistency of the judgment matrix is considered acceptable, since 0.0058 < 0.1, the consistency test of this matrix is proved to be reasonable.
Similarly, the factors of the indicator layer were calculated and ranked in the same way to derive the weight values of the indicator layer and rank them, and the obtained results are shown in Table 2.
CONCLUSION
In conclusion, this paper provides a basis for evaluating, selecting, and trai- ning art teachers in colleges and universities through literature analysis and questionnaire survey, combined with the corresponding comprehensive eva- luation model of NTS. It provides a more accurate direction for improving art teachers’ abilities. We hope to form a complete art teacher evaluation system through further research. More importantly, administrators should institu- tionalize, standardize, and institutionalize the employment and selection of art teachers to improve their non-technical level. These measures will further improve the quality of art education in universities and promote the exchange and learning between disciplines.
ACKNOWLEDGMENT
This work has been supported by the Beijing Municipal Education Commis- sion Science Program (KM202110049001) and the Ideological and Political Education Program for college students of the National Academy of Chinese Theatre Arts (XSCZD2022001, XSCYB2022005).
REFERENCES
Ahlstrom, U. and Arend, L. (2005), “Color usability on air traffic control displays.”
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 49(1), 93–97. Bauerly, M. and Liu, Y. (2006), “Computational modeling and experimental investigation of effects of compositional elements on interface and design aesthetics.” International Journal of Human-Computer Studies, 64(8), 670–682. Bertelsen, O., Petersen, M. G. and Pold, S. (2004), “Aesthetic approaches to human-
computer interaction. DAIMI Report Series, 33(572). Dennis, M. P. (2008), “Perceiving hierarchy through intrinsic color structure.” Visual Communication, 7(2), 199–228. Green, M. (1998), “Toward a perceptual science of multidimensional data visualiza-
tion: Bertin and beyond.” ERGO/GERO Human Factors Science, 8, 1–30. Henry, C. and Peters, J. F. (2010), “Perceptual image analysis.” International Journal of Bio-Inspired Computation, 2(3-4), 271-281. Kankanhalli, M. S., Mehtre, B. M. and Wu, R. K. (1996), “Cluster-based color
matching for image retrieval.” Pattern Recognition, 29(4), 701-708. Kırmızı, B., Colomban, P. and Quette, B. (2010), “On-site analysis of Chinese Cloisonné enamels from fifteenth to nineteenth centuries.” Journal of Raman Spectroscopy, 41(7), 780–790. Li, J., Xue, C., Tang, W. and Wu, X. (2014), “Color saliency research on visual
perceptual layering method.” International Conference on Human-Computer Interaction, 86–97. Peter, B. (2003), “Chromaticity contrast in visual search on the multi-colour user interface.” Displays, 24(1), 39–48. Peters, G. (2011), “Criteria for the Creation of Aesthetic Images for Human-
Computer Interfaces A Survey for Computer Scientists.” International Journal of Creative Interfaces and Computer Graphics (IJCICG), 2(1), 68–98. Quette, B. (1999), “Chinese cloisonne (Illustrated antique objects).” Magazine Antiques, 155(1), 206–213.
Construction of Comprehensive Evaluation Model for Non-Technical Skills 179
Robertson, P. K. (1988), “Visualizing color gamuts: A user interface for the effective use of perceptual color spaces in data displays.” IEEE Computer Graphics and Applications, 8(5), 50–64. Shi, L., Pei, W., Li, J., Miyata, K. and Ma, S. (2021), “Inheritance of Chinese Traditio-
nal Color Culture Based on Modern Human Visual Perception.” 2021 Nicograph International (NicoInt), 106–109. Van Den Broek, E. L., Kisters, P. M. and Vuurpijl, L. G. (2004), “Design guidelines for a content-based image retrieval color-selection interface.” Proceedings of the conference on Dutch directions in HCI, 14). Van Laar, D. L. (2001), “Colour coding with visual layers can provide performance
enhancements in control room displays.” The Second International Conference on Human Interfaces in Control Rooms, Cockpits and Command Centres, 228–233. Van Laar, D.L. (2001), “Psychological and cartographic principles for the production of visual layering effects in computer displays.” Displays, 22(4), 125–135. Wu, J. H. and Yuan, Y. (2003), “Improving searching and reading performance: the
effect of highlighting and text color coding.” Information & Management, 40(7), 617–637. Xue, C., Li, J., Wang, H. and Niu, Y. (2015), “Effects of target and distractor saturations on the cognitive performance of an integrated display interface.” Chinese Journal of Mechanical Engineering, 28(1), 208–216. Zhou, L., Xue, C., Tang, W., Li, J. and Niu, Y. (2013), “Aesthetic evaluation
method of interface elements layout design.” Journal of Computer-Aided Design & Computer Graphics, 25(5), 758–766.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Training, Education, and Learning Sciences
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-35-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional