Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

An Evaluation Method of Art Teachers' Cooperative Communication Ability Based on Task Situation

Beijing Normal University School of Arts & Communication, Beijing, 100091, China; National Academy of Chinese Theatre Arts, Beijing, 100073, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1003163

Abstract

To development an evaluation of the cooperation and communication ability of art teachers in colleges and universities. First, based on the teaching task situation of art discipline, the critical incident technique was introduced to carry out semi-structured interviews. According to the differences in teaching task situations, the art teachers' cooperation ability was scored from three dimensions: establishing and maintaining consensus, taking appropriate actions to solve problems, and maintaining team organization forms. Then, the indicators are set to form a language communication ability evaluation method based on the teaching task scenario. Finally, select front-line teachers with more than five years of teaching experience as the object, design and implement the cooperative communication ability evaluation experiment based on the above evaluation methods. The cooperation and communication ability evaluation experiment can be applied to the quantitative evaluation the performance of art teachers in their post capability. The cooperative communication ability evaluation experiment can provide a new idea for the quantitative and standardized method, and also provide a basis for the next step of developing corresponding evaluation software and tools.

INTRODUCTION

The task-based teaching model is a combination of scenario-based teaching and task-based teaching. Situational teaching mode refers to the creation of visual and vivid scenes based on the content to be taught, which can actively promote students’ self-learning ability and their ability to analyze problems independently. Confidence and cooperation play an important role in lear- ning, and the cooperative learning theory of “teacher-teacher interaction” is used to enhance communication between teachers, advocating that two or more teachers share the task of teaching at the same time in the classroom.

The concept of teacher-teacher interaction is essential for human resources and the importance of teacher-teacher interaction in the educational process (Fu J, 2007). Communication and cooperation is a goal-oriented activity, which is carried out to reach certain pedagogical goals. Art subjects have their educational peculiarities, and communication and cooperation between

© 2023. Published by AHFE Open Access. All rights reserved. 171

teachers and students and teachers is often more than in general subjects, and the ability to cooperate and communicate in different task scenarios is particularly important. While there is more research on the development and study of cooperative communication skills among students, there is less rese- arch on the assessment of cooperative communication skills of art teachers. Based on the cooperative communication ability assessment system, we intro- duce the teaching task context of art disciplines, propose a task context-based method for measuring art teachers’ cooperative and communication abili- ties, and conduct experimental validation through a comprehensive survey method to achieve accurate assessment and dynamic monitoring and deve- lopment prediction of art teachers’ cooperative communication ability levels in task contexts, so as to achieve more efficient and targeted teaching and improve This will enable more efficient and targeted teaching and improve the teaching ability and classroom quality of art teachers.

THEORETICAL RESEARCH Teamwork competence refers to the ability to work as a team member by sharing resources and collaborating with other team members, communica- ting with each other, helping each other and trying to integrate into the team, which eventually makes the whole team complete the task successfully. In cur- rent practice, collaborative problem-solving skills are usually measured in a small, non-standardized way, such as a holistic evaluation by the instructor of each student in the class and described in the student’s end-of-semester rubric. Because most teachers do not receive systematic training in the assessment of collaborative problem-solving skills, and because such holistic evaluati- ons are often subjective and vary in their assessment scales, these rubrics do not accurately reflect students’ collaborative problem-solving skills, whereas teachers’ collaborative communication skills refer to teachers’ ability to pro- vide support to each other in the teaching process and to share knowledge to improve teaching skills. and sharing knowledge, and the ability to collabo- rate in order to improve teaching skills, learn new knowledge, and solve new problems.

So far, there have been some important attempts in empirical studies on large-scale measurement of collaborative problem-solving skills, such as Assessment and Teaching of 21st Century Skills (ATC 21S) (Griffinp, 2012), PISA 2015 Collaborative Problem Solving Skills Assessment (OECD, 2017), and the study of a standardized test prototype implemented by ETS on measu- ring collaborative problem solving skills (Hao J, 2017). All of these large-scale empirical studies used online computer-based virtual collaborative tasks: the PISA 2015 used a human-computer collaborative approach, and the ATC 21S and ETS studies mainly used human-human collaborative approaches.

Based on the above studies, in order to more realistically reflect the col- laborative ability of art teachers in their working state, this test combines special situations with art classroom teaching tasks; to eliminate the uncer- tainty of human-human collaboration, the test adopts the human-computer interaction mode of conversational agents (Yuan J L, 2016); to reduce the

An Evaluation Method of Art Teachers’ Cooperative Communication 173

residuals caused by language ability differences, the test adopts the intera- ction mode of dialogue selection; to optimize the To optimize the structure of the questions, the test was based on the PISA 2015 collaborative problem- solving framework, which measures art teachers’ collaborative skills in 12 dimensions. Finally, with reference to Item Response Theory (IRT) (Hao J G, 2019), the subjects’ collaborative ability was scored by multivariate sco- ring according to the differences in path node selection, and the results were output through the system.

In the process of art teachers’ teaching, collaborative skills are mainly used to solve problems in specific task contexts. In this study, the PISA 2015 asses- sment matrix was used to reflect the subjects’ collaborative problem-solving skills, as shown in Table 1.

Communication is the main expression of team members’ ability to convey information, express emotions, and exchange ideas, and it is generally belie- ved that good communication skills can help to resolve conflicts and enhance

Table 1. PISA 2015 collaborative problem solving test ability matrix.

Table available in the original PDF paper.

Collaborative ability Problem solving ability

(3) Establish and maintain team organization

(2) Take appropriate actions to solve problems

(1) Establishing and maintaining a common understanding

(A) Inquiry and understanding

(A3) Understanding problem-solving roles

(A1) Discovering the perspectives and abilities of team members

(A2) Discovering goals and solving problems Types of cooperative interactions required

(B) Representation and Formation

(B3) Describe roles and team organization

(B2) Identify and describe the tasks that need to be accomplished

(B1) Building shared representations and negotiating problem meaning of the problem

(C) Plan and execute

(C2) Making plans (C3) Compliance with rules of participation

(C1) Communicate with team members the actions that will be or are being performed

(D) Monitoring and feedback

(D2) Monitoring behavior results and evaluation

(D1) Monitoring and refining common understanding

(D3) Monito- ring, providing feedback, and Adapting to team organization and roles

team effectiveness (Zhang Z, 2016). At present, communication skills are mostly measured in the field of education and interpersonal communication in health care, and some tests on communication skills are available in the form of scales and tests. Scales are mainly based on self-assessment or col- league evaluation and are highly subjective and susceptible to a variety of uncertainties. The most widely used test is the word pair matching test in the DXC psychometric system of the Fourth Military Medical University, which tests the ability to understand words accurately. There are international meth- ods such as Cookie theft (Feng X, 2020) and Boston Diagnostic Aphasia Examination (BDAE)(Wang X, 2015), which are mainly used to measure the language ability of patients with brain injury, but their essence is the measure- ment of the subject’s language expression, information transfer, and accurate description. The Boston Diagnostic Aphasia Examination (BDAE) and other methods are mainly used to measure the language ability of brain-damaged patients. The Boston Diagnostic Aphasia Examination (BDAE) is one of the

Table 2. Boston aphasia measurement indexes and scoring criteria.

Table available in the original PDF paper.

Analysis indicators Specific meaning Total number of words Delete the question and response of the main test and count the total number of words. Percentage of incorrect information

Pick out disfluent words

Pick out incorrect expressions in the text. Non- specific nouns, such as stuff, are not specific if you can infer what “stuff” is. then “something” is not a non-specific noun. Percentage of disfluent expressions Pick out words that provide support structure Percentage of words providing support structure

Percentage of repeated content

Pick out the words that are repeated from the pre- vious content. It is not necessary to be precise here that each word is exactly the same as the information expressed earlier. As long as the meaning is the same, it is counted as repetition Percentage of valid image information

The valid information of the picture is selected, and any information that appears in the picture is considered as valid information. Explain the percentage of valid information in the picture

Pick out the words that explain the content unit. Words that explain the content unit include descripti- ons of the characters’ mental activities, descriptions of what happened descriptions of causes, etc. Percentage of irrelevant words

The irrelevant words were selected. The remaining words that could not be classified in the above cate- gories were irrelevant. Information that is not in the picture information is considered irrelevant.

An Evaluation Method of Art Teachers’ Cooperative Communication 175

most widely used diagnostic tests for aphasia in the United States, and a set of standardized scoring criteria has been developed for this test. Therefore, this paper introduces the Boston Diagnostic Aphasia Test and incorporates the art course teaching scenario, presenting the questions by means of pictures and text questions to test the semantic and output abilities in three dimensi- ons: semantic content, lexical content and syntactic complexity, as shown in Table 2.

EXPERIMENTAL DESIGN Art teachers’ collaborative skills are mainly applied in teams of teaching and research groups formed by teachers specializing in different courses. Since each of them has different professional orientations, teaching hours, etc., and treats problems or tasks from different perspectives, additional interactive activities are needed between team members to alleviate conflicts in infor- mation needs and to achieve optimal results through negotiation among all parties (Lai HY, 2020). One of the most necessary aspects of teamwork is the establishment of shared situational awareness (SSA), where everyone in the team maintains the same level of situational awareness (Shi Yusheng, 2018- Wang Yanqing, 2021), and appropriate SSA can form team cognition by per- forming interactive processes (Salmon PM, 2009- Salas E, 2004), while team members need to actively exchange the information necessary to generate team cognition.

Combining the three key aspects of information sharing, interactive negoti- ation, and task completion necessary for art teachers to engage in teamwork, and the PISA 2015 collaborative problem solving framework (Xu S, 2019), the measurement of collaborative competence is divided into three dimensi- ons: building and maintaining consensus, taking appropriate action to solve problems, and establishing and maintaining team organization, presenting ideas, interacting with questions and answers, revising and refining, making plans, assigning work The nine specific indicators are shown in Table 3.

From the beginning to the end of each task, the participant communica- tes, chats, and acts with the conversational agent in multiple rounds, which form a series of specific questions. Each question gave a limited number of options that the test taker chose to determine the path to complete the task, and the end point of one task naturally formed the starting point for the next task. Each test question points to one of nine specific indicators of the Art Teacher Collaboration Competency Assessment. Dynamic situations are arranged during the test, and subjects are required to discuss with conversa- tional agents with reference to contextual changes, class size, lesson schedule, class time, and lesson cycle, and to choose the best instructional option within a limited time frame. The project contained four main task scenarios, with the number of questions chosen from 5-10.

Each option of the test questions points to one of the indicators of art teachers’ collaborative skills assessment, and the different paths are scored through the expert interview method. Finally, the score of the subject’s choice of questions was the total score of the cooperation ability assessment items.

Table 3. Indicators of pilot cooperation ability assessment.

Table available in the original PDF paper.

Primary dimension Secondary dimension Explanatory notes Build and maintain consensus

Give judgments on others’ views and explain them back

Presenting ideas Provides or introduces new ideas, perspectives, suggestions or solutions, clarifies issues Interacting with questions and answers Develop a plan Develop an orderly plan for performing tasks Revise and refine Revise or refine ideas that have been presented Take appropriate action to solve problems

Assign work Rationalize work assignment according to purpose Complete the work Work according to the original plan Build and maintain team organization

Emotional support Expresses emotional support such as comfort, encouragement, and affirmation Information exchange

Exchange of information known to individuals Supervision Supervision of the discussion process and its implementation

ART TEACHERS’ COMMUNICATION ABILITY ASSESSMENT DESIGN Communication is a necessary component of cooperation and an essential way for team interaction, information transfer, and problem negotiation. Communication of art teachers mainly consists of two parts: outputting information and receiving feedback; outputting information mainly refers to the art teachers’ teaching of course knowledge in the course of instruction, while receiving feedback refers to obtaining students’ ideas and opinions in a timely manner. The most important aspect of communication is the authen- ticity of conveying information to the recipient for interpretation (Solari EJ, 2020), i.e., the accuracy of linguistic expressions, where the assessment of lin- guistic competence includes standardized language assessment and narrative discourse assessment, combined with the Boston Aphasia Test dimensions, the initial screening of art teachers’ linguistic competence assessment dimen- sions, and also through the analysis of teaching scenarios as well as expert research found that: Accurate expressions, effective expressions, and flow expressions were found to be the most important in classroom communica- tion. The weights of each index were obtained by expert scoring method, in which the accuracy rate was 40%, the efficiency rate was 35%, and the fluency rate was 25%, as shown in Table 4.

The purpose of the Art Teacher Communication Ability Test is to test the semantic ability and linguistic output of individual art teachers. In this test, four photographs of classroom information (including time, students’ status, teaching space, etc.) of an art course were presented. The participants were asked to describe the content of each picture in detail and to interpret the information in depth (e.g., describe the students’ listening status at this time, etc.). In this process, the subjects were required to keep the content accurate

An Evaluation Method of Art Teachers’ Cooperative Communication 177

Table 4. Indicators of art teachers’ communication ability.

Table available in the original PDF paper.

Specific meaning Weighting

Analysis of indicators

35%

Accuracy rate Accuracy of the information conveyed 40% Efficient Combining key words to extract effective information from the text Fluency rate Fluency of the subject’s verbal expressions 25%

and the language fluent as much as possible. The number of accurate words, effective words and fluent words in each picture description was counted.

The fluency rate was calculated based on the interval of characters, and the fluency rate was calculated from the total number of words to the total time. During the test, the total number of words (N) and description time (T) of each text were recorded, and the number of valid words (n1) and correct words (n2) were extracted to obtain the efficiency (a), accuracy (b) and fluency (c) of the subjects, and finally the final score of the subjects’ language ability assessment was obtained by weighting:

Efficient : an1

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.
Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

T

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

N

2N T

The subjects were all art teachers of the Chinese Academy of Opera, 13 in total, all with more than 5 years of teaching experience. Through data screening and analysis, a total of 13 groups of valid data were retained, and the data were collected from November 2022 to January 2023.

The assessment of cooperation ability relied on the Questionnaire Star platform to design and distribute the questionnaire, and the assessment of communication ability relied on cell phone recordings for voice recognition and text conversion work.

A total of 13 groups of data were collected in this experiment, which were divided into high and low groups according to the scores of subjects’ cooperative ability and communication ability.

Further tests of variance between the two groups of data showed that the scores of cooperative ability of the subjects in the two groups (t = −3. 483, p = 0.010) and communication ability scores (t = −2.625, p = 0.010). 010) and communication ability scores (t = −2.625, p = 0. 034) were significantly different, indicating that the test had good discrimination.

The results were analyzed in summary and the normality test was perfor- med on 13 groups of data and showed significance (p>0.05), implying that the data had normal qualities.

Table 5. The scores of each dimension of art teacher’s cooperative and communicative ability in high and low groups.

Table available in the original PDF paper.

Indicator Name High group Low score Cooperative ability 25.7 13.6 Consensus building and maintenance 11.2 7.7 Take appropriate action to solve problems 12.7 5.2 Establish and maintain team organization 1.8 0.7 Communication skills 6.746 4.23 Accuracy rate 0.724 612 Efficient 0.367 0.301 Flow rate 4.21 3.85

CONCLUSION

In this study, the test of variance for 13 subjects found significant differe- nces between high and low groups, and the measured data possessed good discrimination, proving that building and maintaining consensus, taking appropriate actions to solve problems, and building and maintaining team organization are important indicators for evaluating art teachers’ ability to cooperate, and accuracy, efficiency, and fluency from are important indica- tors for evaluating the level of art teachers’ communication ability. Most of these indicators were positively correlated with the level of competence in this dimension.

This study was conducted in a small scale, and the amount of data was small. Extensive experiments can be conducted to test the validity of the model with more sufficient data, and eventually form a perfect and reliable system for assessing art teachers’ cooperative communication ability, aiming to provide a new idea for the quantitative and standardized assessment of art teachers’ cooperative communication ability, and to provide a basis for the development of corresponding assessment software and tools.

This article is the result of the “Curriculum Thinking and Politics” project of the Chinese Academy of Xiqu - the most beautiful course (general project) “Research on the construction of employment courses in opera schools under the normalization of the new crown epidemic”(project number: KC202210).

ACKNOWLEDGMENT

This work was supported by Ideological Education in Curriculum Rese- arch Project National Academy of Chinese Theatre Arts (Grant Number KC202210).

REFERENCES

Feng X, Yu L, Xu F, et al. (2020). Digital assessment of cognitive communication ability of patients with cerebral microvascular disease based on Cookie Theft test. Journal of Yunnan Normal University (Natural Science Edition), 40(5): 21–24. Fu J. (2007). Professional Development of Teachers: Approaches and Methods.

Shanghai: East China Normal University Press.

An Evaluation Method of Art Teachers’ Cooperative Communication 179

Griffinp, Mcgawb, Care E. (2012). Assessment and teaching of 21st century skills, Dordrecht: Springer. Hao J, He Q. (2019). Cooperative problem-solving ability assessment and its practice.

National Examination, 17(9): 11–21. Hao J, Liu L, Von Davier A. (2017). Initial steps towards a standardized assessment for collaborative problem solving: Practical challenges and strategies. Innovation Assessment of Collaboration: 135–156. Lai H, Chen C H, Zheng P, et. al. (2020). Investigatw the Evolving Context of an

Unstable Approach in Aviation from Mental Model Disconnects with an Agent- based Model. Reliability Engineering & System Safety, 193(1): 1–13. OECD. (2017). PISA 2015 Results: Collaborative Problem Solving. Paris: OECD Publishing, 5. Salas E, Fiore S M. (2004) Team Cognition: Understanding the Factors that Drive

Process and Performance. Gainesville: Research Gate. Gainesville: ResearchGate. 83–106. Salmon PM, Stanton NA, Walker GH, et al. (2009). Measuring Situation Awareness in Complex Systems: Comparison of Measures Study. International Journal of Industrial Ergonomics, 39(4): 490–500. Shi Y, Tian Zhi Q, Huang W. (2018). A Team Situational Awareness Evalua-

tion Method Based on Team and Task Dimensions[J]. Aerospace Medicine and Medical Engineering, 31(2): 216–223. Solari E, Henry A, Mcintyre N, et al. (2020). Testing the Effects of A Pilot Listening Comprehension and Vocabulary Intervention for Individuals with Autism. Research in Autism Spectrum Disorders, 71(10): 101501. Wang X, Feng H, Zhou Y, et al. (2015). A study on the assessment of cognitive

communication ability in Chinese 20–30 year olds. Chinese Rehabilitation Theory and Practice, 21(10): 1133–1137. Wang Y, Zhou S, Liu C. (2021). The influence of flight trainees’ gaze style on team situational awareness. Science, Technology and Engineering, 21(18): 7784–7789. Xu S, Zhang Y, Zheng H. (2019). Exploring the use of collaborative problem-

solving assessment techniques in PISA 2015. Shanghai Educational Research, 39(3): 47–51. Yuan J L, Liu H Y. (2016). Measurement of collaborative problem-solving skills: A perspective on the measurement principles of PISA2015 and ATC21S. Foreign Educational Research, 43(12): 45–56. Zhang Z, Jing X, Bai Y. (2016). Research Progress of Individual Collaboration and

Cooperation Ability. Aerospace Medicine and Medical Engineering, 29(3): 224– 228.

Cover of the proceedings volume Training, Education, and Learning Sciences

Explore the complete digital volume

Training, Education, and Learning Sciences

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-85-8

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →