Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Using Game-Play Metrics to Analyze User Experience of Playing Health-Promoted Somatosensory Video Game

1 Department of Industrial Engineering and Management, Yuan-Ze University, Taiwan ( R.O.C. ) ↗
2 Department of Industrial Engineering and Management, Yuan-Ze University, Taiwan ( R.O.C. ); Assistive Technology Center, Fong-Yung Hospital, Taiwan ( R.O.C. ) ↗
3 Department of Physical Medicine and Rehabilitation, Taichung Hospital, Taiwan (R.O.C.) ↗
4 Graduate School of Geronic, Technology and Service Management, Nan Kai University of Technology, Taiwan (R.O.C.) ↗
Publication year
Digital Object Identifier10.54941/ahfe100485

Abstract

Player experience evaluation is important for understanding game design and deployment problems. Unlike subjective approach which is built upon data expressed by the player, gameplay metrics based approach is based on objective and detailed data collected during gameplay. For Kinect-based somatosensory video games (SVG), gameplay metrics contains the objective gameplay experiences as well as the gameplay performance. Method: This paper analyzes the gameplay metrics collected from two Kinect-based SVG to study their gameplay experiences. The first game is designed for sit-to-stand training. Fifteen chronic rehabilitation patients played the game for two months. Their objective experience is measured using the head movement data collected from the Kinect sensor. Their game performance is the time to complete the game. The second game is designed for upper-limb exercise. Eight elderly played the game 2 times per week for 6 weeks. Their objective experience is measured using the hand movement speed derived from Kinect sensor. The gameplay performance is the gaming scores. Result & Discussion: The objective experiences have moderately strong correlation with the player’s motor ability; whereas the game performance is not related to motor ability. One of the reasons is that the SVG performance is affected not only by the player’s motor ability but also by their psychological status. Conclusion: The objective gameplay experiences of a properly designed Kinect SVG can be used for motor ability evaluation; whereas the gameplay performance can be used for more comprehensive assessment that considers both the motor ability and the psychological motivations.

INTRODUCTION

Somatosensory video game (SVG) merges fun from the gaming side with potential health and wellness benefits from the physical exercise side to let players undertake physical exercise in a pleasant and interesting environment. SVG let players conduct game play and body exercise at the same time. SVG-based rehabilitation training exercise increases the enjoyment, engagement, and adherence of the otherwise bored training process.

Recent emergence of the depth camera Kinect (http://www.xbox.com/Xbox360 ) opens a new opportunity for low-cost, convenient and mark-less body sensing. Kinect-based SVG does not require any hand-held remote control device, but instead it uses 3D depth sensing technology to detect the player’s movement and position. This natural and intuitive interface makes Kinect-based SVG more suitable for stroke patients and elderly suffered from motor deficiency to practice rehabilitation exercises.

Various SVG’s have been developed in the past to motivate elderly to exercise. (Lim et al., 2012) discuss the design of a motivational exercise game whose goal is to increase the activity of elderly players with complex chronic conditions. (Bamidis et al., 2011) describes an innovative e-health service that promotes active aging of elderly players by offering cognitive and physical training. (Graf, Tamanini, & Geissler, 2011) presented a holistic approach to design elderly exergame called ‘Muntermacher’ to motivate seniors enhancing their daily physical activity. (Gerling, Schild, & Masuch, 2010) discuss chances and challenges concluded from the development of an elderly exergame called ‘Silver Balance’. The game implements different balance tasks for elderly players featuring the Nitendo Wii Balance Board to encourage seniors to actively engage in game play. (Khoo, Merritt, & Cheok, 2009) developed an intergenerational exergame called Age Invaders which focuses on physical and social interactions using a mixed reality floor system.

A growing number of studies have confirmed the feasibility of SVG for clinical practices, including post-stroke rehabilitation (Hijmans, Hale, Satherley, McMillan, & King, 2011) and (Duff et al., 2010), training for balance (Yang, Tsai, Chuang, Sung, & Wang, 2008)(Gil-Gómez, Lloréns, Alcañiz, & Colomer, 2011), and motor control exercise for cerebral palsy patients. SVG were also used for motor ability assessment. (Yamada et al., 2011) examined whether Nintendo Wii Fit can be used for fall risk assessment in healthy, community-dwelling elderly. A total of 45 female elderly participated in the study. Their study showed that the nonfaller group present significantly difference in the Basic Step game module (p<0.001) but non-significant different in another game module Ski Slalom. (Wieching et al., 2012) mentioned a knowledge based system for fall prediction and prevention correlates mobility analysis data from sensor monitor and exergame to analysis fall trends. (Liu, Shen, Mei, & Ji, 2013) present a pad game tailored for the Parkinson’s disease diagnosis and rehabilitation. They analyze personal data to detect the potential PD symptoms and monitor the progression of PD patients. (Chang, Lin, Chen, & Lee, 2011) combined SG and wireless sensor for frailty measurement and detection. The system allows users to collect and manage their personal frailty information automatically at home. The authors compare the system with traditional frailty measurement methods. Correlation is found between traditional methods and the game with sensor methods. (Garcia, Felix, & Schoene, 2012) develop a Kinect-based SG to deliver step training while simultaneously measure parameters of stepping performance, which is used to predict fall risks. The authors only described the system but the prediction effect is not described. (Zavala-Ibarra & Favela, 2012) presented a general architecture in which ambient videogames is used for health monitoring. Two ambient casual games were designed to measure arm muscle strength using custom-designed interaction devices. An evaluation experiment with 5 elderly subjects is conducted to assess ease of use and interests. The traditional measures of muscle strength using clinical dynamometer is compared with the video game based approach.

Player experience evaluation is important to understand game design and deployment problems. Effective incorporation of games technology into home, institution, and clinical setting is an important research direction (Lange et al., 2010). Player experience models can be built on three types of data collected from the players (Yannakakis & Togelius, 2011). The subjective experience model is built from data expressed by players. The objective experience model is built from player data obtained from psychological sensors that measure player response, as games can elicit player emotional responses which then affect changes in the player’s physiology. The gameplay-based experience model is built upon data obtained through the interaction between the player and the game.

Most previous research on elderly player experience evaluation is based on subjective approach. (Robinson et al., 2011) compare user’s acceptance and experience through psychological questionnaires for two exergames. After 4-week intervention, no significant differences were found between gaming platforms. But there were significant increases in acceptance and experiences. (Rice et al., 2011) explore the usability and acceptability of three gesture- based exergames. User evaluations with 36 older adults were analyzed using pre- and post- game questionnaires, direct observation and semi-structured group interviews. The results demonstrate that items associated with physical wellbeing were rated highly. (Doyle, Kelly, & Caulfield, 2011) examined the enjoyment and perceived competence with and without interactive visual feedback provided by exergame. The results showed that exercising with game more enjoyable. Additional work is needed to increase player’s confidence in correct execution of the exercise.

The subjective evaluation approaches may not be suitable for elderly or patients who are not able to or unwilling to express their true fillings. Elderly may have difficulty to understand the questionnaire. Elderly at oriental countries are generally not willing to express their true feelings. Subjective techniques are good for understanding the player attitudes but bad at understanding their behaviors (Mandryk, Atkins, & Inkpen, 2006).

Gameplay metrics contain objective and detailed data describing the interactions between the player and the game. Gameplay metrics be collected automatically, frequently, and in non-intrusive manner. Gameplay metrics collected from a Kinect SVG contains data for analyzing objective user experience as well as game performance. Traditional objective experience that requires placing sensors to body parts is difficult to be applied to elderly as they are not willing to wear physiological sensors. Kinect sensor, on the other hand, allows mark-less and convenient skeleton collection.

This paper analyzes gameplay metrics collected from two Kinect-based SVG to study their gameplay experiences. The objective experience models emphasize the affective and the cognitive aspects of playing experience and the gameplay-based experience model focuses on the cognitive and behavioral side (Yannakakis & Togelius, 2011). Therefore, studying gameplay metrics based player experience can help understand how the elderly feel and react when playing a Kinect-based SVG. This can further contribute to better SVG design and deployment.

GAMEPLAY METRICS COLLECTION FOR A KINECT-BASED SIT-TO-STAND SVG Game design

The first Kinect-based SVG is designed for sit-to-stand training of rehabilitation patients. The game is developed using a game authoring tool called Unity 3D (http://unity3d.com ). The player’s body movement is captured by the Kinect depth sensor. The U3D game scene is connected to Kinect using Open NI Kinect SDK (http://75.98.78.94/default.aspx). The interactions between the player and the game objects is programmed using Javascript provided by Unity 3D as well as the built-in physics engine of Unity 3D. Game effects like smoke, exploration, etc. are specified in Unity 3D using its built-in particle systems.

The game, as shown in Figure 1, prompted the player to perform sit-to-stand movement. The game scene contains a stick and ball. When the player stands up, the stick will move upward accordingly. The player had to drive the stick upward until it hits the ball. Each game session requires the player to perform 10 times sit-to-stand movements, i.e., to drive the stick to hit the ball 10 times.

Figure 1. Kinect-based SVG for sit-to-stand training.
Original PDF Figure 1.

Procedure Fifteen chronic rehabilitation patients (M:13, F:2) played the game for two months. They came to the hospital for rehabilitation exercises. During each visit to the hospital, the patients are welcome to play the Kinect game after they finish the physical rehabilitation practices. The gameplay metrics are collected over a two-month period. For the 15 rehabilitation patients who sign the signed consent forms, 2 played the game for only one time; 4 played 2~3 times, 4 played 4~6 visits, 4 played 7~9 times, and one subject played 12 times. At each play time, the player can play as many game sessions as they want. For each game session, the player has to finish 10 times of sit-to-stand movements.

During their first visit, the player’s balance ability is measured. The intrinsic balance ability is measured using the Biodex Balance System (http://www.biodex.com), which gives the overall stability index (SI), APSI (Anterior/Posterior) and MLSI (Medial/Lateral). The performance-based balance ability is measured using the 3- meter timed up and go test (TUG).

Figure 2. Intrinsic and performance-based balance ability measurement (a) Biodex balance system, (b)

Figure available in the original PDF paper.

TUG test.

Player Experience The objective player experience is the total distance of head movement during the 10 sit-to-stand movements. This value is calculated using the head position collected from the Kinect sensor. The game completion time, i.e., time to finish 10 times of sit-to-stand movements, is used as the gameplay performance.

GAMEPLY METRICS ANALYSIS OF A KINECT-BASED UPPER- LIMB EXERCISE SVG Kinect-based Upper-Limb Exercise SVG Design

The second game is a star-touching game for upper-limb exercise. The game scene is designed based on (Burke et al., 2009). Stars show up at a frequency of one star per second and stay for 2 seconds. They randomly showing up along the edge of two concentric circles (highlighted using red line in Figure 3). The elderly players have to move their hand to touch the stars before the star disappears. Each successful touch earns 5 points. Each game session lasts for 1 minute. So for each game session 60 stars will show up and the maximum score the player can earn is 300 points.

The Kinect game is deployed to an elderly nursing center at central Taiwan. Most elderly in the nursing centers at Taiwan suffer from decreased or incomplete motor ability and are forced to sit on wheelchairs. The star-touching SVG is designed to motivate the elderly sitting on wheel to conduct routine, regular upper-limb exercises.

Figure 3. Kinect-based SVG for upper limb exercise.
Original PDF Figure 3.

Procedure Eight elderly (M: 6, F:2) volunteered to play the game. All experimental subjects signed consent forms prior to the experiments. After one game session, the elderly is asked whether they want to play the game again. The elderly can repeat the game play as many times as they want.

The player’s upper arm movement ability is measured using the U/EMMT scale. Level zero means the subject has no movement of upper arm and level 5 means the subject has normal power. For the eight elderly participants in our experiment, 2 are of level-3 ability, which means that they can move their arm against gravity. All the other 6 participants are of level-4 ability, which means that they can move their upper arm against resistance, but the movement is weak.

Player Experience The objective experience is the hand movement speed. The value is calculated using the hand position data collected by the Kinect sensor. The gameplay performance is the star-touching score earned in each game session.

RESULTS

Table 1 shows the correlation between the player experiences and their motor ability for the sit-to-stand SVG. The second column shows that the objective player experience, i.e., the head movement distance in 10 sit-to-stand movements, have moderately strong correlation with the player’s intrinsic stability ability, i.e., the stability indices; but low correlation with the performance based balance measurement, i.e., TUG test.

The third column shows that the game performance, i.e., the time to finish 10 sit-to-stand movements, has mild correlation with the performance based balance measurement, i.e., TUG test; but low correlation with the intrinsic balance measurements, i.e., the stability indices.

Table 1: Player experience vs motor ability for the Kinect-based sit-to-stand SVG.
Original PDF Table 1:

Table 2 shows the correlation between the player experiences and their motor ability for the upper-limb SVG. The objective experience, i.e., the hand movement speed, has a moderately strong correlation with the player’s upper- limb motor ability. On the contrary, the game performance, i.e., the star-touching scores earned during game play, has no relation with the player’s motor ability.

Table 2: Player experience vs motor ability for the Kinect-based upper-limb exercise SVG.
Original PDF Table 2:

Correlation coefficients are Pearson’s

DISCUSSION

The results shown in Table 1 and 2 indicate that the player’s motor ability is related to their objective experiences but not their gameplay performance. One of the reason is because gameplay performance is affected by many psychological factors (Canossa & Cheong, 2011). Canossa and Cheong stated that player behaviour is not homogeneous nor is to be considered a conscious or unconscious expression of personality or intention. The actions of a player are not completely intentional. There are a number of factors that could turn the player’s intentional behaviour into an improvised sequence of actions. Players with good motor ability and motivation should have good game performance. Players with good motor ability but less motivation will turn their intentional behavior to improvised behavior. As such, their gameplay performance becomes lower than expect.

The results can also be explained by the dual flow model proposed in (Sinclair, Hingston, & Masek, 2007), which stated that the design of an exergame like Kinect SVG must consider both the psychological attractiveness and the physiological effectiveness. Attractiveness is a psychological model that balances the player’s skill with the perceived challenge. Effectiveness is the physiological model counterpart that balances the player’s physical exercise skill and the challenge of the exercise on the player’s body. The player’s motor ability determines his/her physical exercise skill, which, in turn, is related to the objective experience measured by the body movement data collected from the Kinect sensor. The gameplay performance, on the other hand, is affected by both the attractiveness and the effectiveness of the Kinect SVG. Gameplay performance is affected not only by the physical exercise skill but also the psychological model. Therefore, it is not related to the player’s motor ability.

Both (Sun & Lee, 2013) and (Yamada et al., 2011) found that the player’s gameplay performance will be affected by the challenging feeling presented in the game. (Sun and Lee 2013) further showed that the player’s intrinsic physical exercise skill will not be affected by the challenging feeling presented in the game. This finding is consistent with the results shown in this paper. Even if the player has low motivation and has lower game performance, their objective experience calculated from the Kinect sensor is still related to their motor ability.

CONCLUSIONS

This paper analyzes the gameplay metrics of two Kinect-based somatosensory games to understand the relationships between the user experiences and their motor ability. Two types of player experiences can be measured from Kinect gameplay metrics. The objective user experience is measured using the body movement data captured by the Kinect depth sensor. The gameplay performance is recorded during gameplay. For the sit-to-stand game, the objective experience measured from Kinect data is closely related to the intrinsic balance ability; whereas the game performance based player experience is closely related to the performance based balance ability. For the upper arm exercise game, the objective experiences measured using Kinect data is closely related to the upper arm muscle power; whereas the gameplay performance is not related to the player motor ability. This effect can be explained by the fact that the gameplay performance is influenced not only by the player’s motor ability but also the player’s psychological feelings and their gameplay skills.

ACAKNOWLEDGEMENT This work is partially supported by National Science Council, Taiwan, R.O.C. under contract NSC 101-2627-E-155- 001.

REFERENCE Bamidis, P. D., Konstantinidis, E. I., Billis, A., Frantzidis, C., Tsolaki, M., Hlauschek, W., … Pattichis, C. S. (2011). A Web

services-based exergaming platform for senior citizens: the Long Lasting Memories project approach to e-health care. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Vol. 2011, pp. 2505–9). doi:10.1109/IEMBS.2011.6090694 Burke, J. W., McNeill, M., Charles, D., Morrow, P., Crosbie, J., & McDonough, S. (2009). Serious Games for Upper Limb Rehabilitation Following Stroke. In 2009 Conference in Games and Virtual Worlds for Serious Applications (pp. 103– 110). Ieee. doi:10.1109/VS-GAMES.2009.17 Canossa, A., & Cheong, Y.-G. (2011). Between Intention and Improvisation : Limits of Gameplay Metrics Analysis and

Phenomenological Debugging. In Proceedings of DiGRA 2011 Conference: Think Design Play. Chang, Y.-C., Lin, C.-C., Chen, C.-C., & Lee, R.-G. (2011). A Home-Based Frailty Detection System Using Wireless Sensor Technology with Multimedia Interactive Games. In 2011 Fifth FTRA International Conference on Multimedia and Ubiquitous Engineering (pp. 119–122). Ieee. doi:10.1109/MUE.2011.32 Doyle, J., Kelly, D., & Caulfield, B. (2011). Design Considerations in Therapeutic Exergaming. In Proceedings of the 5th

International ICST Conference on Pervasive Computing Technologies for Healthcare (pp. 389–393). Ieee. doi:10.4108/icst.pervasivehealth.2011.246115

Duff, M., Chen, Y., Attygalle, S., Herman, J., Sundaram, H., Qian, G., … Rikakis, T. (2010). An adaptive mixed reality training system for stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(5), 531–41. doi:10.1109/TNSRE.2010.2055061 Garcia, J. A., Felix, K., & Schoene, D. (2012). Exergames for the elderly : Towards an embedded Kinect - based clinical test of

falls risk. Studies in Health Technology and Informatics, 178, 51–57. doi:10.3233/978-1-61499-078-9-51 Gerling, K. M., Schild, J., & Masuch, M. (2010). Exergame Design for Elderly Users : The Case Study of SilverBalance. In 7th International Conference on Advances in Computer Entertainment Technology, ACE 2010 (pp. 66–69). Gil-Gómez, J.-A., Lloréns, R., Alcañiz, M., & Colomer, C. (2011). Effectiveness of a Wii balance board-based system (eBaViR)

for balance rehabilitation: a pilot randomized clinical trial in patients with acquired brain injury. Journal of Neuroengineering and Rehabilitation, 8(30). doi:10.1186/1743-0003-8-30 Graf, H., Tamanini, C., & Geissler, L. (2011). Muntermacher-“ Think and Move” Interface and Interaction Design of a Motion- Based Serious Game for the Generation Plus. Lecture Notes in Computer Science, 6766(Part 2), 149–158. Hijmans, J. M., Hale, L. a., Satherley, J. a., McMillan, N. J., & King, M. J. (2011). Bilateral upper-limb rehabilitation after stroke

using a movement-based game controller. The Journal of Rehabilitation Research and Development, 48(8), 1005. doi:10.1682/JRRD.2010.06.0109 Khoo, E. T., Merritt, T., & Cheok, A. D. (2009). Designing physical and social intergenerational family entertainment.

Interacting with Computers, 21(1-2), 76–87. doi:10.1016/j.intcom.2008.10.009 Lange, B. S., Requejo, P., Flynn, S. M., Rizzo, a a, Valero-Cuevas, F. J., Baker, L., & Winstein, C. (2010). The potential of virtual reality and gaming to assist successful aging with disability. Physical Medicine and Rehabilitation Clinics of North America, 21(2), 339–56. doi:10.1016/j.pmr.2009.12.007 Lim, J. H., Zhan, A., Ko, J., Terzis, A., Szanton, S., & Gitlin, L. (2012). A Closed-Loop Approach for Improving the Wellness of

Low-Income Elders at Home Using Game Consoles. IEEE Communications Magazine, 50(1), 44–51. Liu, S., Shen, Z., Mei, J., & Ji, J. (2013). Parkinson ’ s Disease Predictive Analytics through a Pad Game Based on Personal Data.

International Journal of Information Technology, 19(2), 1–17. Mandryk, R. L., Atkins, M. S., & Inkpen, K. M. (2006). A continuous and objective evaluation of emotional experience with interactive play environments. In Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI ’06 (p. 1027). Montreal, Quebec, Canada: ACM Press. doi:10.1145/1124772.1124926 Rice, M., Wan, M., Foo, M.-H., Ng, J., Wai, Z., Kwok, J., … Teo, L. (2011). Evaluating gesture-based games with older adults

on a large screen display. In Proceedings of the 2011 ACM SIGGRAPH Symposium on Video Games - Sandbox ’11 (pp. 17–24). New York, New York, USA: ACM Press. doi:10.1145/2018556.2018560 Robinson, J., Schaik, P. P. Van, Macsween, A., Dixon, J., Martin, P. D., & Instrumentation, B. (2011). User-Acceptance and Flow in Two Gaming Platforms Used for Exercise. In International Conference on Virtual Rehabilitation 2011 (pp. 26– 27). Sinclair, J., Hingston, P., & Masek, M. (2007). Considerations for the design of exergames. In GRAPHITE 2007 (Vol. 1, pp.

289–296). Sun, T., & Lee, C.-H. (2013). An Impact Study of the Design of Exergaming Parameters on Body Intensity from Objective and Gameplay-Based Player Experience Perspective, Based on Balance Training Exergame. PloS One, 8(7), e69471. Wieching, R., Kaartinen, N., DeRosario, H., Baldus, H., Eichberg, S., Drobics, M., & Delbaere, K. (2012). A new approach for

personalized fall risk prediction & prevention: tailored exercises, unobtrusive sensing & advanced reasoning. Journal of Aging and Physical Activity, 120–124. Yamada, M., Aoyama, T., Nakamura, M., Tanaka, B., Nagai, K., Tatematsu, N., … Ichihashi, N. (2011). The reliability and preliminary validity of game-based fall risk assessment in community-dwelling older adults. Geriatric Nursing (New York, N.Y.), 32(3), 188–94. doi:10.1016/j.gerinurse.2011.02.002 Yang, Y.-R., Tsai, M.-P., Chuang, T.-Y., Sung, W.-H., & Wang, R.-Y. (2008). Virtual reality-based training improves

community ambulation in individuals with stroke: a randomized controlled trial. Gait & Posture, 28(2), 201–6. doi:10.1016/j.gaitpost.2007.11.007 Yannakakis, G. N., & Togelius, J. (2011). Experience-Driven Procedural Content Generation. IEEE Transactions on Affective Computing, 2(3), 147–161. doi:10.1109/T-AFFC.2011.6 Zavala-Ibarra, I., & Favela, J. (2012). Ambient Videogames for Health Monitoring in Older Adults. 2012 Eighth International

Conference on Intelligent Environments, 27–33. doi:10.1109/IE.2012.56

Cover of the proceedings volume Advances in Human Aspects of Healthcare

Explore the complete digital volume

Advances in Human Aspects of Healthcare

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2093-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →