Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The Best Fit Framework for Human Computer Interaction Research ‒ Is it possible?

Department of Computer Science and Informatics, University of the Free State, Bloemfontein, South Africa ↗
Publication year
Digital Object Identifier10.54941/ahfe1005463

Abstract

The Best Fit Framework, originally proposed by Carroll et al. (2013) to synthesize qualitative data has been successful to conduct a review of the literature to produce models or frameworks for decision making and health behaviours. While successful in health behaviours, it has not been implemented within Human-Computer Interaction before. This paper aims to convey knowledge, experiences, and recommendations towards the use of the Best Fit Framework to synthesize data in the field of Human-Computer Interaction. The Best Fit Framework involves various stages. The first two stages run simultaneously and involve identifying relevant frameworks, models, or theories, using the BeHEMoTh (Behaviour of Interests; Health Context; Exclusions; Models, Theories, Frameworks) search technique, and to identify relevant primary research studies with qualitative evidence, using the SPIDER (Setting/Population; Phenomenon of Interest; Design, Evaluation, Research) search technique. The selected theories, frameworks, or models are then reduced to key elements and used as themes in the new framework. These themes are interpreted and compared to new or similar types of themes across the literature, found with the SPIDER technique. New identified themes are incorporated to create an updated framework. After the framework is created, it is tested as a final part of the synthesis process. To apply the Best Fit Framework in the field of Human-Computer Interaction, the researchers expanded the context of BeHEMoTh. The researchers sought to also include the prevention or minimization of Cybersickness. The inclusion of quantitative primary research studies as part of the SPIDER technique was added, as the original SPIDER technique focused on qualitative studies which assisted in expanding the pool of primary research studies. The last change addressed how the framework synthesis was tested. Rather than only revisiting evidence to create and explore relationships, the researchers evaluated as part of the newly created CyPVICS framework in a real-world case study to determine validity. The case study compared the usability and user experience of two immersive Virtual Reality navigation methods, namely touch controllers vs. omnidirectional treadmill, in the training of nursing students. In conclusion, the Best Fit Framework proved to be adaptable and useful in Human-Computer Interaction research.

© 2024. Published by AHFE Open Access. All rights reserved. 47

48 Botha and de Wet

INTRODUCTION

The Best Fit Framework was originally proposed by Carroll et al. (2013) as a refined method to synthesize qualitative data for creating frameworks. While the framework has been successful as a synthesis methodology to conduct reviews of the literature to produce models or frameworks for decision making and health behaviours, it has not been implemented within Human-Computer Interaction before.

This paper aims to convey knowledge, experiences, and recommendations towards the use of the Best Fit Framework to synthesize data in the field of Human-Computer Interaction. The process conveyed in this paper was part of a larger study and was used to create the CyPVICS framework (Botha and De Wet, 2024) to reduce or eliminate cybersickness (CS) during immersive virtual simulation.

THE BEST FIT FRAMEWORK The Best Fit Framework (Carroll et al., 2013) uses the current developments in the qualitative data synthesis methodology to conduct a review of the literature to produce models or frameworks (Carroll, Booth and Cooper, 2011). This framework also provides a way to test, strengthen or improve existing models, for example, applying it to a different population. To use the framework, a relevant framework, theory, or model must be identified. The selected theory, framework or model is then reduced to certain key elements which are used as themes in the new framework. The themes are then interpreted by the reviewer and compared to new or similar types of themes across the literature (Carroll et al., 2013).

The Best Fit Framework (Carroll et al., 2013) involves various stages. Each stage depicts the application of a certain strategy or methodology. The first two stages run simultaneously. They include identifying relevant frameworks, conceptual models, or theory publications, using the BeHEMoTh technique, and to identify relevant research studies with qualitative evidence, using the SPIDER technique. These stages of the Best Fit Framework will be discussed in detail in the following sections, starting with the BeHEMoTh technique.

BeHEMoTh Technique The BeHEMoTh was created as a structured way to identify and specify relevant models or theories for a literature review. The BeHEMoTh consists of four parts, namely behaviour of interest (Be), health context (H), exclusions (E) and models or theories (MoTh) (Booth and Carroll, 2015).

Within the BeHEMoTh, there are various steps that must be adhered to so as to create an accurate priori framework, theory, or model. Once all publications have been identified, a priori framework must be constructed by means of thematic analysis. Thematic analysis is a method to analyse qualitative data sets by identifying recurring themes in different literature or data (Nowell et al., 2017). The themes that are identified from the BeHEMoTh are combined and integrated to create the priori framework, theory or model (Carroll et al., 2013). The next stage of the Best Fit Framework (Carroll et al., 2013), which runs concurrently with the BeHEMoTh, is the SPIDER technique.

The Best Fit Framework for Human Computer Interaction Research – Is it Possible? 49

SPIDER Technique The SPIDER technique was created in response to limitations of the PICO (population/problem, intervention/exposure, comparison, and outcome). The SPIDER technique was meant to generate search terms easier than the PICO, while the results were also easier to manage (Cooke, Smith and Booth, 2012). SPIDER involves five components, namely the setting/population (S), the phenomenon of interest (PI), the design (D), evaluation (E) and the research type (R). The setting/population refers to the sample or population and to the environment or setting to which the research is bound, for example, undergraduate nursing students in South Africa. The phenomenon of interest refers to the phenomenon or event being investigated or researched, for example, cybersickness (CS). The design, evaluation and research type refer to the research design, the methods of evaluation and the type of research, for example, case studies using mixed methods, and usability and UX evaluation (Carroll et al., 2013).

To assist with the validity of the themes and elements in the priori framework, theory or model, the research studies obtained from the SPIDER technique must be appraised to determine their quality. This is done by synthesising the models, theories, and frameworks. Once the BeHEMoTh and SPIDER analyses are completed and the results are synthesised, the results from the research studies must be coded to that of the framework, theory or model to determine the validity, along with the themes and elements (Carroll et al., 2013).

With the coding completed, the last steps of the Best Fit Framework (Carroll et al., 2013) must be applied, namely, to create and add new themes based on aspects that could not be added in the initial creation, and to finally test the framework, theory or model.

Finalising of Framework, Theory or Model After the initial coding and thematic analysis, another analysis is necessary to include the results which could not be coded against the framework, theory, or model. These newly identified themes then need to be incorporated into the framework, theory, or model to produce an updated version. This version must finally be analysed to explore relationships between themes. It is then tested by means of exploring dissonance and the impact of variability. By testing the framework, theory or model, biases are reduced, for example, certain themes might be absent, which might actually be very important and relevant (Carroll et al., 2013). It is important to note that evaluation or testing of a framework generally is a cyclical process and can be done on a continuous basis to improve the created framework (Hevner, 2007; Gregor and Hevner, 2017).

THE BEST FIT FRAMEWORK IN HUMAN COMPUTER INTERACTION To apply the Best Fit Framework in the field of Human-Computer Interaction, the researchers expanded the context of BeHEMoTh. While the original technique included the health context only, the researchers

50 Botha and de Wet

sought to also include the prevention or minimization of Cybersickness (CS) (as the context of our study). The second change was the inclusion of quantitative primary research studies as part of the SPIDER technique. While the original SPIDER technique focused on qualitative studies only, the researchers included tried and tested primary studies in both qualitative and quantitative research. This assisted in expanding the pool of primary research studies, while also increasing the validity of a created framework. The third change addressed how the framework synthesis was tested. Rather than only revisiting evidence to create and explore relationships, the researchers tested (evaluated) a part of the newly created CyPVICS framework (i.e. sensory mismatch/conflict theory and improved models of interaction) in a real- world case study to determine validity. The case study involved comparing the usability and user experience (UX) of two immersive Virtual Reality navigation methods, namely touch controllers vs. omnidirectional treadmill, in the training nursing students (Botha and de Wet, 2024).

Question and Search Strings The first step in the Best Fit Framework (Carroll et al., 2013) was to determine the terms that would form part of the search strings for both the BeHEMoTh and the SPIDER techniques, from the main research question, namely: “Which determinants should form part of a framework for designing immersive virtual clinical simulations to prevent or minimise cybersickness?”. Although the main research question was the same for both the BeHEMoTh and SPIDER techniques, the way in which the search terms were extracted from this question, differed for these two techniques. The BeHEMoTh and the SPIDER techniques were also linked to the subsidiary research questions, namely:

  • (BeHEMoTh): “What are the common and unique constructs in existing frameworks, theories and models to prevent/minimise cybersickness in immersive virtual clinical simulation?”
  • (SPIDER): “Which aspects affect/influence cybersickness during immersive virtual clinical simulation?”

From the main research question, the first term that was identified for use in both the BeHEMoTh and the SPIDER techniques was immersive virtual clinical simulation, as it represented the application area of this study. Since alternative terms are used in literature for ‘immersive’ virtual clinical simulation (VCS), for example, simply VCS, or VRS (when in fact the authors are referring to immersive VCS), all related terms were used. Thus, all the research papers that focused on immersive VCS were included.

The second term was cybersickness (or CS, as it is often referred to), as it was the phenomenon in question for this study. The third term was determinants, which related to either constructs (subsidiary research question one) or aspects (subsidiary research question two). For all the terms, synonyms were also sourced from various literature studies where the study focused on immersive virtual reality in the clinical context. The synonyms assisted in obtaining the best possible results and to include as many literature

The Best Fit Framework for Human Computer Interaction Research – Is it Possible? 51

sources as possible related to the three identified terms (see Table 1 for all synonyms used in the search strings).

Once the applicable terms and synonyms were identified, the technique- specific approach could follow. The terms not indicated in Table 1 were excluded since they were not applicable to this study.

Table 1. Search terms and synonyms for BeHEMoTH and SPIDER techniques.

Table available in the original PDF paper.

Search Terms

Question Extract and Applicable Technique Immersive Virtual clinical simulation (SIPDER and BeHEMoTh) Virtual clinical simulation (VCS) (Foronda, Godsall and Trybulski, 2013) Virtual Reality Simulation (VRS) (Jenson and Forsyth, 2012; Dubovi, Levy and Dagan, 2017) Virtual Simulation (VS) (Aebersold, Tschannen and Bathish, 2012) Clinical Virtual Simulation (CVS) (Padilha et al., 2018) active HMD-based virtual reality (Arcioni et al., 2019) virtual reality experiences with head-mounted displays (Arttu, 2018) Virtual Reality (Elwardy et al., 2020) HMD-based virtual reality (Chang, Kim and Yoo, 2021) immersive content (Melo, Vasconcelos-Raposo and Bessa, 2018) VR simulation (Servotte et al., 2020) Cybersickness (SIPDER and BeHEMoTh)

Cybersickness (CS) (Wang et al., 2019; Weech et al., 2019) Virtual reality induced motion sickness (VRIMS) (Li et al., 2021) Virtual reality induced symptoms and effect (VRISE) (Kemeny et al., 2017) Visually induced motion sickness (VIMS) (Wang et al., 2019; Weech et al., 2019) Simulator Sickness (Duzmanska, Strojny and Strojny, 2018) Motion Sickness (Joseph, Browning and Jiang, 2020) Virtual Reality Sickness (Chang, Kim and Yoo, 2020) Aspects (Relates to subsidiary research question 1) (SPIDER) All terms relate to well-known methods and techniques in primary research studies

Models, Theories, Frameworks (Carroll et al., 2013)

Quantitative Qualitative Mixed method Case Study Interview Views Attitudes Focus group Experiment Opinions Constructs (Relates to subsidiary research question 2) (BeHEMoTh)

The search strings for the BeHEMoTh and the SPIDER techniques were sent to an information specialist at the researcher’s institutional library to conduct the search and return the results. The information specialist used the following databases to search for the literature: Academic Search Ultimate, Africa-Wide Information, APA PsycArticles, APA PsycInfo, CAB Abstracts, CINAHL with Full Text, ERIC, GreenFILE, Health Source - Consumer Edition, Health Source: Nursing/Academic Edition, Humanities Source Ultimate, MEDLINE, OpenDissertations, SPORTDiscus with Full

52 Botha and de Wet

Text, Scopus, WoS. The technique-applicable search strings and their results will be discussed in the sections to follow, starting with the BeHEMoTh technique.

BeHEMoTh Search String and Results Two separate searches were done to obtain literature on models, theories or frameworks that could assist in compiling the CyPVICS Framework (Botha and De Wet, 2024). Both search strings had no date delimiter. The first search string for the BeHEMoTh technique was ((Be AND H AND MoTh) NOT E) and can be seen in Table 2.

Table 2. First BeHEMoTh search string used for this study.

Table available in the original PDF paper.

First BeHEMoTh search string Be Cybersickness OR Virtual reality induced motion sickness OR Virtual reality induced symptoms and effect OR visually induced motion sickness OR Simulator Sickness OR Motion Sickness OR Virtual Reality Sickness

AND

H Virtual clinical simulation OR Virtual Reality Simulation OR Virtual Simulation OR Clinical Virtual Simulation OR active HMD-based virtual reality OR virtual reality experiences with head-mounted displays OR Virtual Reality OR HMD-based virtual reality OR immersive content OR VR simulation

AND MoTh Model OR Theory OR Framework

NOT

E Case Study OR Interview OR Views OR Attitudes OR Focus group OR Experiment OR Opinions OR Animal Model

From this search string six results were found; however, none were classified as a model theory or framework. Because the literature was lacking in the context of VCS, the search string was broadened by reducing it to (Be and MoTh), as can be seen in Table 3. More models, theories, and frameworks were obtained, although they were not necessarily in a VCS context. Even though these models, theories or frameworks were from different disciplines, CS was still applicable here. Consequently, they were evaluated for possible inclusion into the CyPVICS Framework (Botha and De Wet, 2024).

The Best Fit Framework for Human Computer Interaction Research – Is it Possible? 53

Table 3. Broadened search string for BeHEMoTh applicable to this study.

Table available in the original PDF paper.

Second BeHEMoTh search string Be Cybersickness OR Virtual reality induced motion sickness OR Virtual reality induced symptoms and effect OR visually induced motion sickness OR Simulator Sickness OR Motion Sickness OR Virtual Reality Sickness

AND MoTh Model OR Theory OR Framework

From the second BeHEMoTh search string, a total of 1057 results were obtained after automatic deduplication. Once the search was completed, all the abstracts and titles were evaluated to determine whether they conformed for inclusion into this framework. The literature was evaluated based on the criteria in Table 3.

During the process of abstract and title evaluation, a total of 1050 titles and abstracts were excluded due to not being applicable to the terms identified for the BeHEMoTh technique. Once the abstract and title evaluations were completed, the full papers were sourced and analysed to determine their eligibility. From the seven papers, references were analysed to determine whether there were more articles that could be included as part of the ancestry search. After the ancestry search, a total of nine titles and abstracts were included, which brought the total full text articles that had to be reviewed for inclusion in the construction of the initial CyPVICS Framework for this study, to 12.

SPIDER Search String and Results The SPIDER technique was used to find primary research studies without limiting the date when the study was published. The search strategy for the first SPIDER technique (S AND PI AND DER) and the search string can be seen in Table 4. From the SPIDER technique, a total of 417 results were obtained after automatic deduplication. Once the search was completed, all the abstracts and titles were evaluated to determine whether they conformed to the research question from which the SPIDER technique search terms were derived.

During abstract and title evaluation, a total of 350 titles and abstracts were excluded as they were not applicable to the terms identified for the SPIDER technique in Table 4. Once the abstract and title evaluations were completed, the full papers were sourced and analysed to determine their eligibility. From the 57 papers, references were analysed to determine whether there were articles which could be included as part of the ancestry search. After an ancestry search, 13 titles and abstracts were included, which brought the article count to 70 full text articles that were reviewed for possible inclusion as primary research studies. In total, 39 studies conformed to the requirements. The data for the primary research studies were extracted into a table format. The raw data can be viewed table at https://doi.org/10.38140/ ufs.22955402.v1.

54 Botha and de Wet

Table 4. SPIDER search string used for this study.

Table available in the original PDF paper.

SPIDER search string S Virtual clinical simulation OR Virtual Reality Simulation OR Virtual Simulation OR Clinical Virtual Simulation OR active HMD-based virtual reality OR virtual reality experiences with head-mounted displays OR Virtual Reality OR HMD-based virtual reality OR immersive content OR VR simulation

AND

PI Cybersickness OR Virtual reality induced motion sickness OR Virtual reality induced symptoms and effect OR visually induced motion sickness OR Simulator Sickness OR Motion Sickness OR Virtual Reality Sickness

AND

DER Quantitative OR Qualitative OR Mixed method OR Case Study OR Interview OR Views OR Attitudes OR Focus group OR Experiment OR Opinions

CONCLUSION

In conclusion, the Best Fit Framework proved to be adaptable and useful in Human-Computer Interaction research. This involved changes to the original Best Fit Framework, which included expanding the context, incorporating quantitative research studies, and implementing a usability and UX evaluation in a real-world case study.

REFERENCES

Aebersold, M., Tschannen, D. and Bathish, M. (2012) ‘Innovative Simulation Strategies in Education’, Nursing Research and Practice, 2012, pp. 1–7. Available at: https://doi.org/10.1155/2012/765212. Arcioni, B. et al. (2019) ‘Postural stability predicts the likelihood of cybersickness in

active HMD-based virtual reality’, Displays, 58(April 2018), pp. 3–11. Available at: https://doi.org/10.1016/j.displa.2018.07.001. Arttu, T. (2018) Effect of Visual Realism on Cybersickness in Virtual Reality.

University of Oulu. Available at: https://jultika.oulu.fi/files/nbnfioulu- 201802091218.pdf. Booth, A. and Carroll, C. (2015) ‘Systematic searching for theory to inform systematic reviews: Is it feasible? Is it desirable?’, Health Information and Libraries Journal, 32(3), pp. 220–235. Available at: https://doi.org/10.1111/hir. 12108. Botha, B. S. and De Wet, L. (2024) ‘CyPVICS: A framework to prevent or minimise

cybersickness in immersive virtual clinical simulation’, Heliyon, 10(8), p. e29595. Available at: https://doi.org/10.1016/j.heliyon.2024.e29595. Carroll, C. et al. (2013) ‘“best fit” framework synthesis: Refining the method’, BMC Medical Research Methodology, 13(1). Available at: https://doi.org/10. 1186/1471-2288-13-37. Carroll, C., Booth, A. and Cooper, K. (2011) ‘A worked example of “best fit”

framework synthesis: A systematic review of views concerning the taking of some potential chemopreventive agents’, BMC Medical Research Methodology, 11. Available at: https://doi.org/10.1186/1471-2288-11-29.

The Best Fit Framework for Human Computer Interaction Research – Is it Possible? 55

Chang, E., Kim, H. T. and Yoo, B. (2020) ‘Virtual Reality Sickness: A Review of Causes and Measurements’, International Journal of Human-Computer Interaction, pp. 1658–1682. Available at: https://doi.org/10.1080/10447318. 2020.1778351. Chang, E., Kim, H. T. and Yoo, B. (2021) ‘Predicting cybersickness based on user’s

gaze behaviors in HMD-based virtual reality’, Journal of Computational Design and Engineering, 8(February), pp. 728–739. Available at: https://doi.org/10.1093/ jcde/qwab010. Cooke, A., Smith, D. and Booth, A. (2012) ‘Beyond PICO: The SPIDER tool for qualitative evidence synthesis’, Qualitative Health Research, 22(10), pp. 1435–1443. Available at: https://doi.org/10.1177/1049732312452938. Dubovi, I., Levy, S. T. and Dagan, E. (2017) ‘Now I know how! The learning process of medication administration among nursing students with non-immersive desktop virtual reality simulation’, Computers and Education, 113, pp. 16–27. Available at: https://doi.org/10.1016/j.compedu.2017.05.009. Duzmanska, N., Strojny, P. and Strojny, A. (2018) ‘Can simulator sickness be

avoided? A review on temporal aspects of simulator sickness’, Frontiers in Psychology, 9(NOV). Available at: https://doi.org/10.3389/fpsyg.2018.02132. Elwardy, M. et al. (2020) ‘Evaluation of Simulator Sickness for 360

  • Videos on an HMD Subject to Participants’ Experience with Virtual Reality’, in 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, pp. 477–484. Available at: https://ieeexplore.ieee.org/document /9090447/. Foronda, C., Godsall, L. and Trybulski, J. A. (2013) ‘Virtual Clinical Simulation: The

State of the Science’, Clinical Simulation in Nursing. Available at: https://doi.org/ 10.1016/j.ecns.2012.05.005. Gregor, S. and Hevner, A. R. (2017) ‘Positioning and Presenting Design Science Research for Maximum Impact’, MIS Quarterly [Preprint]. Available at: https: //doi.org/10.2753/MIS0742-1222240302. Hevner, A. R. (2007) ‘A Three Cycle View of Design Science Research’, Scandinavian

Journal of Information Systems, 19(2), pp. 87–92. Jenson, C. E. and Forsyth, D. M. (2012) ‘Virtual reality simulation: Using threedimensional technology to teach nursing students’, CIN - Computers Informatics Nursing, 30(6), pp. 312–318. Available at: https://doi.org/10.1097/NXN.0b013e 31824af6ae. Joseph, A., Browning, M. H. E. M. and Jiang, S. (2020) ‘Using Immersive Virtual

Environments (IVEs) to Conduct Environmental Design Research: A Primer and Decision Framework’, Health Environments Research and Design Journal, 13(3), pp. 11–25. Available at: https://doi.org/10.1177/1937586720924787. Kemeny, A. et al. (2017) ‘New VR navigation techniques to reduce cybersickness’, IS and T International Symposium on Electronic Imaging Science and Technology, pp. 48–53. Available at: https://doi.org/10.2352/issn.2470-1173.2017.3.ervr-097. Li, H. et al. (2021) ‘The effect of navigation method and visual display on distance

perception in a large-scale virtual building’, Cognitive Processing [Preprint], (0123456789). Available at: https://doi.org/10.1007/s10339-020-01011-4. Melo, M., Vasconcelos-Raposo, J. and Bessa, M. (2018) ‘Presence and cybersickness in immersive content: Effects of content type, exposure time and gender’, Computers and Graphics (Pergamon), 71, pp. 159–165. Available at: https://do i.org/10.1016/j.cag.2017.11.007.

56 Botha and de Wet

Nowell, L. S. et al. (2017) ‘Thematic Analysis: Striving to Meet the Trustworthiness Criteria’, International Journal of Qualitative Methods, 16(1), pp. 1–13. Available at: https://doi.org/10.1177/1609406917733847. Padilha, J. M. et al. (2018) ‘Clinical Virtual Simulation in Nursing Education’,

Clinical Simulation in Nursing, 15, pp. 13–18. Available at: https://doi.org/10. 1016/j.ecns.2017.09.005. Servotte, J. C. et al. (2020) ‘Virtual Reality Experience: Immersion, Sense of Presence, and Cybersickness’, Clinical Simulation in Nursing, 38, pp. 35–43. Available at: https://doi.org/10.1016/j.ecns.2019.09.006. Wang, Y., Chardonnet, J. R. and Merienne, F. (2019) ‘VR sickness prediction

for navigation in immersive virtual environments using a deep long short term memory model’, 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings, pp. 1874–1881. Available at: https://doi.org/10.1109/VR .2019.8798213. Weech, S., Kenny, S. and Barnett-Cowan, M. (2019) ‘Presence and cybersickness in virtual reality are negatively related: A review’, Frontiers in Psychology, 10(FEB), pp. 1–19. Available at: https://doi.org/10.3389/fpsyg.2019.00158.

Cover of the proceedings volume Human Interaction and Emerging Technologies (IHIET 2024)

Explore the complete digital volume

Human Interaction and Emerging Technologies (IHIET 2024)

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-33-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →