Martin Thissen, Thi Ngoc Diep Tran, Ben Joel Schönbein, Ute Trapp, Barbara Esteve Ratsch, Beate Egner, Romana Piat, Elke Hergenröther (2024). Enhancing Canine Musculoskeletal Diagnoses: Leveraging Synthetic Image Data for Pre-Training AI-Models on Visual Documentations. In: Health Informatics and Biomedical Engineering Applications. AHFE Open Access, vol. 142, pp. 60–69. https://doi.org/10.54941/ahfe1005071
The examination of the musculoskeletal system in dogs is a challenging task in veterinary practice. The careful diagnosis as well as the evaluation of very complex findings is getting increasingly important. Therefore, a novel method has been developed that enables efficient documentation of a dog's condition through a visual representation. However, since the visual documentation is new, there is no existing training data. The objective of this work is therefore to mitigate the impact of data scarcity in order to develop an AI-based diagnostic support system that can provide veterinarians with accurate predictions. To this end, the potential of synthetic data that mimics realistic visual documentations of diseases for pre-training AI models is investigated. Specifically, this work explores whether pre-training an AI model with synthetic data can improve the overall accuracy of canine musculoskeletal diagnoses.We propose a method for generating synthetic image data that mimics realistic visual documentations. Initially, a basic dataset containing three distinct classes is generated, followed by the creation of a more sophisticated dataset containing 36 different classes. Both datasets are used for the pre-training of an AI model, adapting it to the domain of visual documentations. Subsequently, an evaluation dataset is created, consisting of 250 manually created visual documentations for five different diseases. This dataset, along with a subset containing 25 examples, serves as the basis for evaluating the efficacy of pre-training an AI model on synthetic data.The obtained results on the evaluation dataset containing 25 examples demonstrate a significant enhancement of approximately 10% in diagnosis accuracy when utilizing generated synthetic images that mimic real-world visual documentations. However, these results do not hold true for the larger evaluation dataset containing 250 examples, indicating that the advantages of using synthetic data for pre-training an AI model emerge primarily when dealing with few examples of visual documentations for a given disease. This implies that the use of synthetic data may not be necessary for diseases with many visual documentation examples.Overall, this work provides valuable insights into mitigating the limitations imposed by limited training data through the strategic use of generated synthetic data, presenting an approach applicable beyond the canine musculoskeletal assessment domain.
Enhancing Canine Musculoskeletal Diagnoses: Leveraging Synthetic Image Data 61
INTRODUCTION
In the ever-evolving landscape of artificial intelligence (AI) applications, effec- tively utilizing AI models in practical use cases where training data is limited remains a challenging problem. Particularly, in veterinary medicine where the availability of huge annotated datasets is limited (Appleby and Basran 2022). One promising way to address this challenge is the exploration of few-shot learning (FSL) models, which aim to efficiently learn representations when provided only with a handful of training examples. However, state-of-the-art FSL models almost always aim to transfer knowledge gained from large-scale source tasks, utilizing huge datasets, to smaller-scale target tasks with limited training data availability (Hu et al., 2022). This approach, however, becomes problematic when the target task (e.g., canine musculoskeletal diagnosis) significantly differs from the available source tasks with huge datasets.
Therefore, an alternative strategy involves augmenting training data with synthetic examples mimicking the limited available data (Trabucco et al., 2023, Burg et al., 2023). This way, Deep Learning-based classification models can potentially thoroughly learn the characteristics of canine musculoskeletal diagnoses. For this reason, we investigated in this work whether pre-training an AI model with synthetic data can improve the overall accuracy of canine musculoskeletal diagnoses.
To this end, we developed a method to generate synthetic images mimick- ing visual documentations of canine musculoskeletal diagnoses. For this, we first generated a basic dataset containing three distinct classes, followed by creating a more sophisticated dataset with 36 classes. Both datasets were used to pre-train a deep learning classification model, adapting it to the domain of canine musculoskeletal diagnosis. To assess the pre-training effectiveness, we manually created an evaluation dataset containing 250 visual documenta- tions of five diagnoses (50 examples per diagnosis). Additionally, we explored the effectiveness with minimal data, using a subset of 25 examples from the evaluation dataset (5 examples per diagnosis).
Our results indicate that the use of synthetic data is not universally benefi- cial. Although a significant enhancement of approximately 10% in diagnosis accuracy was achieved in a few-shot scenario (5 examples per diagnosis), this improvement did not hold true for the larger evaluation dataset with 250 examples (50 examples per diagnosis). This indicates that the advantages of using synthetic data for pre-training an AI model emerge primarily when dealing with a few examples of visual documentations for a given disease.
GROWING DEMAND FOR EFFECTIVE DOCUMENTATION IN VETERINARY MEDICINE In response to the growing demand for evidence-based veterinary care, central veterinary hospitals are emerging as hubs where specialized prac- titioners collaborate across disciplines. This trend mirrors developments in human medicine, enabling the integration of specialties such as Physical and Rehabilitative Medicine (PRM) into veterinary practice (European Physical and Rehabilitation Medicine Bodies Alliance, 2018). As the importance of PRM in veterinary medicine rises, addressing specific indications becomes imperative.
The escalating prevalence of sportive dogs requiring preventive and demanding soft tissue injury treatments underscores the need for enhanced PRM competence. Additionally, the aging pet population, expected to reach 30–40%, requests a proactive senior medicine approach (Fortney, 2012). The rising concern of obesity, a prevalent medical issue in dogs, further empha- sizes the urgency for comprehensive and effective veterinary interventions (Ratsch et al., 2022).
PRM, a discipline focused on preventing, diagnosing, treating, and rehabil- itating patients with disabilities, plays a vital role in understanding the inter- dependencies between bodily functions and structures. The PRM practitioner collaborates with various specialists to deliver accurate diagnoses, requir- ing strong communication skills and interdisciplinary cooperation (European Physical and Rehabilitation Medicine Bodies Alliance, 2018).
Diagnosing PRM-related issues involves detailed anamnesis, physical examinations, and functional assessments, encompassing pain, joint mobility, muscle power, and more. In complex cases, additional circulatory func- tion testing may be necessary. A thorough diagnosis forms the foundation for a multimodal treatment plan, emphasizing the need for close interdisci- plinary collaboration and effective communication (European Physical and Rehabilitation Medicine Bodies Alliance 2018).
After this, we needed to determine which classification model should be used to determine the effectiveness of using synthetic data to improve the overall accuracy of canine musculoskeletal diagnoses. To this end, we examined several classification models in an exploratory analysis. After an exploratory analysis, we decided to proceed with the EfficientNet V2-S model (Tan and Le, 2021) in subsequent experiments.
We then fine-tuned the EfficientNet V2-S model (pre-trained on the Ima- geNet dataset (Deng et al., 2009)) on the two generated datasets, the one with three classes and the one with 36 classes. In this way, we obtained two fine-tuned EfficientNet V2-S models, which we abbreviate as V2-S-3 and V2-S-36. To evaluate the effectiveness of the synthetic data, we fur- ther fine-tuned these models on the training data of our evaluation dataset. Here we have fine-tuned the V2-S-3, the V2-S-36, and the original V2-S model. We also fine-tuned the V2-S model to be able to compare it with the V2-S-3 and V2-S-36 models. This allows us to assess the effectiveness of pre-training an AI model with synthetic data on the overall accuracy of canine musculoskeletal diagnoses.
RESULTS
To assess the effectiveness of using synthetic data, we compared two models trained with synthetic data to a model trained without synthetic data. More specifically, we fine-tuned all three models V2-S, V2-S-3, and V2-S-36 (intro- duced in the section before) using an evaluation dataset, which we manually created. We also extracted a subset of the evaluation dataset to evaluate the effectiveness of synthetic data in a scenario where very few examples are available.
Table 1. Comparison between one model trained without synthetic data (V2-S) and two models trained with synthetic data (V2-S-3 and V2-S-36). The comparison of the smaller evaluation dataset containing 25 training images is shown in the left table. The right table shows comparison results on the larger evaluation dataset containing 250 training images.
Table available in the original PDF paper.
Model Accuracy (%) Model Accuracy (%) V2-S 80 V2-S 96 V2-S-3 90 V2-S-3 94 V2-S-36 80 V2-S-36 96
Table 1b) shows the results for the larger evaluation dataset containing 250 training and 50 test images. Here, contrastingly the V2-S and V2-S-36 models achieve a better accuracy with 96% compared to the V2-S-3 model with 94%. This contrast in achieved accuracy for the smaller and larger eval- uation datasets needs to be further investigated in future work. Overall, it can be seen that more training images lead to higher accuracy. Also, the observa- tion in Table 1a) that using synthetic images can lead to an overall improved accuracy of canine musculoskeletal diagnoses does not hold true for the larger evaluation dataset, as can be seen in Table 1b).
DISCUSSION
In this work, we investigated whether pre-training an AI model with synthetic data can improve the overall accuracy of canine musculoskeletal diagnoses. Our results indicate that the use of synthetic data is beneficial in scenarios where only limited data is available, specifically five examples per class in our case. For an evaluation dataset with 50 training images per class, the use of synthetic data did not lead to an improvement in the accuracy of canine musculoskeletal diagnoses.
Enhancing Canine Musculoskeletal Diagnoses: Leveraging Synthetic Image Data 67
Nevertheless, our results show that even simplistic synthetic images can lead to improved accuracy. Furthermore, we believe that the integration of exist- ing scientific literature in the form of text data as prior knowledge about canine musculoskeletal diagnoses is promising.
document a canine’s condition has the potential to advance canine muscu- loskeletal assessment. Our investigation of mitigating data scarcity through the use of synthetic data led to a notable improvement of around 10% in diagnosis accuracy on a smaller evaluation dataset, showcasing the potential of pre-training an AI model with generated synthetic images. While the posi- tive impact on the diagnosis accuracy by using synthetic data diminished for a larger dataset, our findings still show the potential of synthetic data, espe- cially when training data is very limited. This research not only contributes to advancing AI-based diagnostic support systems for canine health but also offers insight into broader applications facing data scarcity.
REFERENCES
Appleby, R. B., & Basran, P. S. (2022). Artificial intelligence in veterinary medicine.
Journal of the American Veterinary Medical Association, 260(8), pp. 819–824. Burg, M. F., Wenzel, F., Zietlow, D., Horn, M., Makansi, O., Locatello, F., & Russell, C. (2023). A data augmentation perspective on diffusion models and retrieval. Unpublished. Retrieved January 19, 2024, from https://arxiv.org/pdf/2304.10253. pdf. Cornwall, A. (1992). Body mapping in health RRA/PRA. In RRA Notes, 16(July),
pp. 69–76. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Li Fei-Fei. (2009). ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. European Physical and Rehabilitation Medicine Bodies Alliance. White Book on
Physical and Rehabilitation Medicine (PRM) in Europe. In European Journal of Physical and Rehabilitation Medicine (2018), 54(2), pp. 125–155 and 230–260. Flutter - Build apps for any screen (2024). Retrieved January 19, 2024, from https: //flutter.dev/. Fortney, W. D. (2012). Implementing a successful senior/geriatric health care program
for veterinarians, veterinary technicians, and office managers. In The Veterinary clinics of North America: Small animal practice, 42(4), pp. 823–834. Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., & Greenspan, H. (2018). GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 321, pp. 321–331. Galloway, S. S. (2011). How to document a dental examination and procedure using
a dental chart. In American Association of Equine Practitioners (pp. 35–49). Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative Adversarial Nets. In Advances in Neural Information Processing Systems. Gysin, M., Tschuor, F., Schweizer, D. E., Nett, C., & Geissbühler, U. (2019).
Stand der Digitalisierung in Schweizer Tierarztpraxen. In Schweizer Archiv für Tierheilkunde, 161(5), pp. 307–317. Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, pp. 6840–6851. Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). ImageNet Classification with
Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems.
Enhancing Canine Musculoskeletal Diagnoses: Leveraging Synthetic Image Data 69
Lustgarten, J. L., Zehnder, A., Shipman, W., Gancher, E., & Webb, T. L. (2020).
Veterinary informatics: forging the future between veterinary medicine, human medicine, and One Health initiatives-a joint paper by the Association for Veteri- nary Informatics (AVI) and the CTSA One Health Alliance (COHA). In JAMIA open, 3(2), pp. 306–317. Ratsch, B. E., Levine, D., Wakshlag J. J. (2012). Clinical Guide to Obesity and Nonherbal Nutraceuticals in Canine Orthopedic Conditions. In The Veterinary clinics of North America: Small animal practice, 52(4), pp. 939–958. Sheng, Z., Bollig, E., Granick, J., Zhang, R., & Beaudoin, A. (2022). Canine Par-
vovirus Diagnosis Classification Utilizing Veterinary Free-Text Notes. In 2022 IEEE 10th International Conference on Healthcare Informatics, pp. 614–615. Tan, M., & Le, Q. (2021). EfficientNetV2: Smaller Models and Faster Training. In Proceedings of the 38th International Conference on Machine Learning, pp. 10096–10106. Tarr, J., & Thomas, H. (2011). Mapping embodiment: Methodologies for represent-
ing pain and injury. In Qualitative research, 11(2), pp. 141–157. Trabucco, B., Doherty, K., Gurinas, M., & Salakhutdinov, R. (2023). Effective Data Augmentation With Diffusion Models. In ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Health Informatics and Biomedical Engineering Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-18-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional