Juan Uguña, Juan Pablo Zúñiga, Andrea Barrera, Andrea Argudo, Alexandra Bermeo, Omar Alvarado Cando (2022). Developing a software for the affective image tests using eye tracking technology. In: Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems. AHFE Open Access, vol. 22. https://doi.org/10.54941/ahfe1001052
The development and implementation of software that consists of digitizing a psychological test, in synchronization with the acquisition of eye biosignals is presented. The test uses 32 affective images from the IAPS and OASIS sets, implemented in the PsychoPy software, allowing the establishment of presentation times for visual stimuli, colors, texts, etc. For the calibration and acquisition of data, such as fixation times, first gaze, saccadic movements, and heat maps, the use of Gazepoint Control and Gazepoint Analysis is necessary. The software was validated by a group of psychologists with a population of 3 children, 3 adolescents, and 4 adults, with a duration of 8 minutes. From the pilot test, a survey was conducted among psychologists, obtaining a score of 9.22 out of 10, resulting in the validation of the software.
Eye tracking technology has presented an increasing application on different areas (e.g. neuromarketing, psychology, neuroscience, education), due to its flexibility to adapt to the research fields. Most academic studies, which apply this type of system, the biases or attention and retention times, are evaluated. They are important because show the areas of interest or the area which caught the most the attention of the participant (Smith and Waterman, 1981; Tobiidynavox, 2007). Moreover, human vision is a process that integrates the eyes and brain through a network of receptor neurons and specialized cells. It depends on several cognitive and emotional processes (e.g. attention, memory, information processing), they facilitate the use of cognitive resources, to pay conscious attention to tasks that require it (Pons and Martinez, 2004).
There are different instruments or tests for psychological assessment, there are projective (Esposito et al. 2017), behavioral (Bou et al. 2020), and psycho-affective (Sari et al.2017) tests, among others. Those are paper based, which takes longer to apply and score. The eye- tracking technology allows to adapt multiple assessment tools, such as the affective imagery set IAPS (Lang and Bradley, 1997) and OASIS (Kurdi et al. 2017). Different studies with eye-tracking provide a unique way to observe human visual attention allocation, identifying where a person looks (Laurens et al. 2019). This device is used in multiple studies within the neuropsychological field, for pathologies such as Alzheimer's (Laurens et al. 2019), post- traumatic stress (Milanak et al. 2018), autism spectrum disorder (Venker et al. 2019), aphasia (Mack et al. 2018), eating disorders (Van Ens et al. 2019), depression and anxiety (Sears et al. 2019), cerebral palsy (Andrade et al. 2020), aggressiveness (Gómez et al. 2020), attentional bias (Argudo et al. 2021), inclusive education (Alvarado et al. 2019; Jara et al. 2018), and others.
This paper proposes a software for the digitization of a set of affective images IAPS and OASIS, previously validated (Andrade, 2020; Gomez, 2020), allowing the acquisition of data. The acquired data are sample of the images, repetition loops, data storage, synchronization, with the acquisition of bio-signals, modification of visual stimuli, among other features. In addition, eye bio-signal data, such as saccadic movements, fixation times, and heat maps are obtained.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
DEVELOPMENT SOFTWARE The experiment was developed using Psychopy, Gaze Control, and Gaze Analyzer. The operation of the software (see Figure 1) starts by checking if it meets two initial requirements, i) Gazepoint Control is running and ii) the computer has access to the Internet. If neither of them is not met, the program will restart. Then, the program will start all the necessary sub- processes, one of them is the implementation of the TCP/IP protocol for the connection with Gazepoint Control. Afterwards, the eye tracker calibration is necessary, so the user must be at a distance of 60 cm. Next, the screen and zones of interest will be reviewed, which, if defective, will restart the process. At the end of the calibration process, the software connects to the database to start the loop, in which the bio-signals are acquired via Gazepoint Analysis.
Figure 1. Diagrams Case
Figure available in the original PDF paper.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
SOFTWARE IMPLEMENTATION
Figure 2, shows the block diagram of PsychoPy. At the Start, there is the welcome and the program Start key. Then, the Calibration phase, where the communication socket is established between Psychopy and Gazepoint Control, through a TCP / IP protocol, also, libraries such as socket, visual, event, core, among others, are exported. Then, there is the Beginning, where the connection with Gazepoint Analysis is established and other libraries subprocess are imported. Lastly, the Loop starts. It begins with the preparation for data collection through Gazepoint Analysis, then an image called "Fixation cross" is presented, to focus the participant's gaze. After that, visual stimuli is presented for five seconds, and the result is stored in the desired database.
Figure 2: Diagram of Psychopy
Figure available in the original PDF paper.
Figure 3, presents the Gazepoint Control operation block diagram. The start block means that the software should be running at the time of starting, then the TCP / IP link with PsychoPy is made, and it waits for the activation key. Once it is entered, the calibration is carried out, ending the Gazepoint Control function.
Figure 3: Diagram of the Gazepoint Control operation
Figure available in the original PDF paper.
Figure 4, shows the block diagram of how Gazepoint Analysis works. For the program to run, it is necessary for PsychoPy to be in the Start block, to start a Gazepoint Analysis
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 execution thread. Then, the data is acquired through PsychoPy. Finally, Gazepoint Analysis disconnects from Psychopy.
Figure 4: Diagram of how Gazepoint Analysis works
TESTING AND EVALUATION OF RESULTS
METHODS
The software was evaluated by students of the Neuropsychology master's degree, a pilot test with the following population: 3 children, 3 adolescents, and 4 adults. A total of 16 visual stimuli or slides were presented, which contains 2 affective images, one denoting a positive stimulus and the other denoting a negative stimulus. Each slide has a presentation time of 5 seconds, followed by the fixation cross for one second. In total, the test consisted on 32 images, 16 from the IAPS set and 16 from the OASIS set. For the test, the eye tracker was set at a frequency of 60 Hz, and a distance of 60 cm to the patient. Below, Figure 5 presents the results of the pilot test, heat maps, and saccades (Fig. 5a). Here, three colors are the most noticeable. Red represents the area in which the participant fixed the longest, the blue area represents the place furthest from the main fixation points, while the yellow area is the average between the main point and the farthest ones. Figure 5b, shows the saccades and the fixation times of each one, allowing to recreate the order in which, the participant fixed the eyesight.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
Figure 5: Heat Map (a) and Saccades and fixation times (b)
RESULTS
For the analysis of the obtained results, 3 areas of interest were taken as reference: positive, negative, and other. The images were validated in positive and negative stimuli, and are related to the aforementioned areas; while the last area refers to any excess place on the screen or outside of it. From the obtained data, a classification was made, as follows: The evaluation showed that 45% of the children eyesight movements, were directed to the positive image; also, 42% in adolescents and 43% in adults, so the values are similar. Meanwhile, the percentage of glances that were directed to the negative image was 34% in children, 44% in adolescents and 30% in adults. These last results, show that there is a noticeable change in children and adolescents vs. adults. Finally, the percentage of glances that were lost on or off the screen were 21% for children, 14% for adolescents and 27% for adults, which shows that there was a higher rate of loss in adults (Figure 6).
Figure 6: Bart chart of the first fixation.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
The initial sight of the user was mostly at the positive image. In the group of children, it is observed that the difference between the positive and negative image is only 4 percentage points, while in the group of adolescents the difference is more noticeable with 66 points, lastly, in the group of adults, the difference decreased to 24 points (Figure 7).
First Fixation
Positive Image
0 10 20 30 40 50 60 70 80 90
Children Teenagers Adults
Negative Image
Figure 7: Bar chart of the first fixation.
Figure available in the original PDF paper.
For the validation of the software, a survey was applied to 5 psychologists. It consisted of 5 questions: i) Ease of Software Management, ii) Software functionality, iii) Execution time (6 min - 8 min), iv) Main needs covered and v) Collection / Interpretation of Data (Excel- Videos). Through these questions, a quantitative rating was made with a maximum value of 10 points, thus having 2 points per question. Each question has at least 4 possible answers, which denote the degree of satisfaction when using the software, so the value of each answer will decrease according to the degree of conformity. In the end, the score of each participant was averaged to obtain a definitive score. The average value of each question and the final validation score are detailed below (Table 1):
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
Table 1: Average validation score
CONCLUSIONS
The synchronization to obtain the data through the eyetracker-computer communication was successful. It was not only functional data obtained, from psychological tests, but also providing a tool that allows a deeper analysis by having the data organized in spreadsheets, which can be linked to various statistical analysis programs. In addition, it was possible to confirm that changes in illumination did not affect the calibration of the eye tracker; however, it was evident that in patients who wore glasses there was a certain problem at the time of calibration.
REFERENCES
Alvarado, O. Jara, B. Barzallo P. Torres, H. (2018). A Software Based on Eye Gaze to Evaluate Mathematics in Children with Cerebral Palsy in Inclusive Education. In: AHFE 2018. Advances in Intelligent Systems and Computing, vol 794. Springer, Cham. Andrade, K. Alvarado, O. Ortega, P. Cobos, M. (2020). An Eye Tracking System to Per-
form the Wechsler Intelligence Children-Fifth Edition (WISC-V) Test: A Case Study. AHFE 2019. Advances in Intelligent Systems and Computing, vol 972. Springer, Cham. Argudo, A. Alvarado, O. Calderon, C. Buele, F. Ortega, P. Cobos-Cali, M. (2021). Measure of the Attentional Bias in Children Using Eye Tracking During a Psycho-logical Test. AHFE 2021. Advances in intelligent systems and computing. Springer, Cham. Bou, C. Adelantado, M. Beltran, M. Moliner, D. (2020). Association between Health-Related
Physical Fitness and Self-Rated Risk of Depres-sion in Adolescents: Dados Study. International Journal of Environmental Research and Public Health, 17(12), 4316. Esposito, M. Messina, A. Monda, V. Bitetti, I. Salerno, F. Precenzano, F. Carote-nuto, M.
(2017). The Rorschach Test Evaluation in Chronic Childhood Migraine: A Prelimi- nary Multicenter Case–Control Study. Frontiers in Neurology. Gómez, G. Ortega, P. Alvarado, O. Cobos, M. Artega, S. (2020). Eye Tracking in the Diagnosis of Aggressive Behaviors and Emotions: A Systematic Review of the Literature. AHFE 2019. Advances in Intelligent Systems and Computing, vol 972. Springer, Cham.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
Jara, B. Barzallo, P. Alvarado, O. (2018). A software based on eye gazed to evaluate vowels in children with cerebral palsy in inclusive education. In: IEEE 2018. IEEE ANDESCON. Kurdi, B. Lozano, S. Banaji, M. (2017). Introducing the Open Affective Standardized Im-
age Set (OASIS). Behav Res, 49, 457–470. Lang, P. Bradley, M. (1997). International Affective Picture System (IAPS): Technical Manual and Affective Ratings. Laurens, B. Planche, V. Cubizolle, S. Declerck, L. Dupouy, S. Formaglio, M. Ti-son, F.
(2019). A Spatial Decision Eye-Tracking Task in Patients with Prodromal and Mild Alzheimer’s Disease. Journal of Alzheimer’s Disease, (1), 1–1. Mack, J. Wei, A. Gutierrez, S. Thompson, C. (2016). Tracking sentence comprehension: Test-retest reliability in people with aphasia and unimpaired adults, Neurolinguistics, vol. 40, pp. 98–111. Milanak, M. Judah, R. Berenbaum, H. Kramer, A. Neider, M. (2018). PTSD symptoms and
overt attention to contextualized emotional faces: Evidence from eye-tracking. Psy- chiatry Res. Vol. 269. 408-413. Pons, A. Martínez, F. (2004). Fundamentos de visión binocular. Valencia: Maite Simon. Sarı, S. Bilek, G. Çelik, E. (2017). Test anxiety and self-esteem in senior high school students: a cross-sectional study. Nordic Journal of Psychiatry, 72(2), 84–88. Sears, C. Quigley, L. Fernandez, A. Newman, K. Dobson, K. (2019). The reliability of at-
tentional biases for emotional images measured using a free-viewing eye-tracking para- digm. Behav. Res. Methods, vol. 51, no. 6, pp. 2748–2760. Smith, T. Waterman, M. (1981). Identification of Common Molecular Subsequences. J. Mol.
Biol. 147, 195—197. Tobiidynavox (2007). Available in: https://www.tobiidynavox.com/es/productos/software/. Van Ens, W. Schmidt, U. Campbell, C. Roefs, A. and Werthmann, J. (2019). Test-retest reliability of attention bias for food: Robust eye-tracking and reaction time indices, Appe- tite, vol. 136, no. November 2018, pp. 86–92. Venker, C. Pomper, R. Mahr, T. Edwards, J. Saffran, J. Ellis Weismer, S. (2019). Com-
paring Automatic Eye Tracking and Manual Gaze Coding Methods in Young Children with Autism Spectrum Disorder. Autism Research.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-7923-8988-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional