Animal affective-state analysis using Convolutional Neural Networks: Methods, datasets, applications, challenges, and future directions

Use of CNN in animal affective-state analysis

Authors

DOI:

https://doi.org/10.62310/liab.v6i1.454

Keywords:

Animal emotion states, Animal emotion recognition, Deep learning, Animal welfare, Convolutional Neural Networks, Facial expression analysis

Abstract

Animal emotions are inner affective and bodily states that make certain physiological, behavioral, and survival responses to environmental changes. Comprehension of the animal behavior and emotional states is pivotal a step to improve human-animal relationship and enhance animal welfare. Convolutional Neural Networks (CNNs) are among the popular methodologies to analyze the animal-emotion related data. However, CNNs face challenges such as dataset scarcity, inter-breed morphological variability, and challenges in associating subtle behavioral micro expressions with emotion labels. CNNs help predict animal emotions via successive convolutional layers, images pass through these layers and automatically extract features like eyebrow, mouth curvature, and classify them into different emotional states. Various methods, including AlexNet, ResNet, EfficientNet, MobileNet, have been commonly used for feature extraction and classification. Leveraging the potential of CNN in image analysis and data generation, it can be applied to animal emotion-state analysis. This review outlines the fundamental issues of emotion classification and information retrieval by CNN techniques; and critically evaluates the available literature on the CNN applications in animal affective-state analysis, advocates research gaps, and proposes future research direction. Combining CNNs with data augmentation technique can greatly enhanced model accuracy and prediction, and addresses the issues with small training datasets. Various methods for emotion detection and very few available datasets used by researchers or to used by other researchers for future research are there. The use of CNN-based approaches will greatly benefit the researchers to enhance their studies and help them develop and implement real-world applications.

Metrics

Metrics Loading ...

References

Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. (2018). Sanity checks for saliency maps. Advances in Neural Information Processing Systems 31: 9505–9515. https://doi.org/10.48550/arXiv.1810.03292

Ahmad M, Zhang W, Smith M, Brilot B, Bell M. (2023). Real-time livestock activity monitoring via fine-tuned Faster R-CNN for multiclass cattle behaviour detection. 2023 IEEE 14th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), New York, USA. Pp. 805–811. https://doi.org/10.1109/UEMCON59035.2023.10316066

Bhave A, Hafner A, Bhave A, Gloor PA. (2024). Unsupervised canine emotion recognition using momentum contrast. Sensors 24(22): 7324. https://doi.org/10.3390/s24227324

Bhave A, Kieson E, Hafner A, Gloor PA. (2025). Identifying novel emotions and wellbeing of horses from videos through unsupervised learning. Sensors 25(3): 859. https://doi.org/10.3390/s25030859

Boneh-Shitrit T, Feighelstein M, Bremhorst A, Amir S, Distelfeld T, Dassa Y, Yaroshetsky S, Riemer S, Shimshoni I, Mills DS, Zamansky A. (2022). Explainable automated recognition of emotional states from canine facial expressions: the case of positive anticipation and frustration. Scientific Reports 12(1): 22611. https://doi.org/10.1038/s41598-022-27079-w

Bremhorst A, Sutter NA, Würbel H, Mills DS, Riemer S. (2019). Differences in facial expressions during positive anticipation and frustration in dogs awaiting a reward. Scientific Reports 9: 19312. https://doi.org/10.1038/s41598-019-55714-6

Briefer EF, Sypherd CCR, Linhart P, Leliveld LM, Padilla de la Torre M, Read ER, Guérin C, Deiss V, Monestier C, Rasmussen JH, Špinka M, Düpjan S, Boissy A, Janczak AM, Hillmann E, Tallet C. (2022). Classification of pig calls produced from birth to slaughter according to their emotional valence and context of production. Scientific Reports 12(1): 3409. https://doi.org/10.1038/s41598-022-07174-8

Chattopadhyay A, Sarkar A, Howlader P, Balasubramanian VN. (2018). Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, USA. Pp. 839–847. https://doi.org/10.1109/WACV.2018.00097

Chen HY, Lin CH, Lai JW, Chan YK. (2023). Convolutional neural network-based automated system for dog tracking and emotion recognition in video surveillance. Applied Sciences 13(7): 4596. https://doi.org/10.3390/app13074596

Cheng WK, Leong WC, Tan JS, Hong ZW, Chen YL. (2022). Affective recommender system for pet social network. Sensors 22(18): 6759. https://doi.org/10.3390/s22186759

Chicco D, Jurman G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21: 6. https://doi.org/10.1186/s12864-019-6413-7

Cohen J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1): 37–46. https://doi.org/10.1177/001316446002000104

Corujo LA, Kieson E, Schloesser T, Gloor PA. (2021). Emotion recognition in horses with convolutional neural networks. Future Internet 13(10): 250. https://doi.org/10.3390/fi13100250

Coutant M, Villain AS, Briefer EF. (2024). A scoping review of the use of bioacoustics to assess various components of farm animal welfare. Applied Animal Behaviour Science 275: 106286. https://doi.org/10.1016/j.applanim.2024.106286

Descovich KA, Wathan J, Leach MC, Buchanan-Smith HM, Flecknell P, Farningham D, Vick SJ. (2017). Facial expression: an under-utilised tool for the assessment of welfare in mammals. ALTEX 34(3): 409–429. https://doi.org/10.14573/altex.1607161

Dhenia RNK, Sridhar R, Sharma J, Kanani IJ, Banerjee D, Sharma G. (2025). Using MobileNetV2 for real-world applications: effective animal image classification. Global Conference on Information Technology and Communication Networks (GITCON), Belagavi, India. Pp. 1–5. https://doi.org/10.1109/GITCON65266.2025.11376889

Du X, Shan W, Wang Y, Ye T, Zhuo J. (2023). Dog emotion classification with transfer learning. Research project report available on Julia(Xi) Du Personal Website. https://superjuliadu.com/report/Dog-Image-Classification-Final-Report.pdf

Feighelstein M, Ricci-Bonot C, Hasan H, Weinberg H, Rettig T, Segal M, Distelfeld T, Shimshoni I, Mills DS, Zamansky A. (2024). Automated recognition of emotional states of horses from facial expressions. PLOS One 19(7): e0302893. https://doi.org/10.1371/journal.pone.0302893

Feighelstein M, Shimshoni I, Finka LR, Luna SP, Mills DS, Zamansky A. (2022). Automated recognition of pain in cats. Scientific Reports 12(1): 9575. https://doi.org/10.1038/s41598-022-13348-1

Gähwiler S, Bremhorst A, Tóth K, Riemer S. (2020). Fear expressions of dogs during New Year fireworks: A video analysis. Scientific Reports 10: 16035. https://doi.org/10.1038/s41598-020-72841-7

Gilhotra K, Yadav V, Diwakar M, Lohumi Y, Singh P, Roka S. (2025). Few shot pet emotion recognition using prototypical networks with ResNet34 backbone. In: Goar V, Gajrani J, Shin J, Stowik A, editors, Deep Learning and Visual Artificial Intelligence. ICDLAI 2025. Lecture Notes in Networks and Systems 2097. Springer Nature Switzerland. Pp. 368–378. https://doi.org/10.1007/978-3-032-32476-4_27

Guo C, Pleiss G, Sun Y, Weinberger KQ. (2017). On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning 70: 1321–1330. https://doi.org/10.48550/arXiv.1706.04599

Hernández-Vanegas LA, Rodríguez-Romo S. (2026). Exploring canine emotions: A transfer learning and 3DCNN-based study for small databases. IEEE Transactions on Affective Computing 17(3): 3824-3831. https://doi.org/10.1109/TAFFC.2026.3652076

Jamalpur B, Kothandaraman D, Sathwika V, Pranathi V. (2024). Pet facial emotion recognition using convolutional neural networks. In: Reddy VS, Wang J, Chetti P, Reddy KTV, editors, Soft Computing and Signal Processing, ICSCSP 2024. Lecture notes in Network Systems 1221. Springer Nature Singapore. Pp. 207–216. https://doi.org/10.1007/978-981-96-0924-6_17

Kapoor S, Narayanan A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns 4(9): 100804. https://doi.org/10.1016/j.patter.2023.100804

Kaufman S, Rosset S, Perlich C, Stitelman O. (2012). Leakage in data mining: formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data 6(4): 15. https://doi.org/10.1145/2382577.2382579

Kim JH, Colaco SJ, Poulose A, Neethirajan S, Han DS. (2023). Advancing pig welfare assessment: Introducing the SSPD-PER method for objective and reliable pig emotion recognition. Available on Preprints. https://doi.org/10.20944/preprints202307.1510.v1

Koo TK, Li MY. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine 15(2): 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

Kopczyński M, Czubenko M. (2026). Enhancing human-dog interaction through deep learning and explainable AI. Scientific Reports 16: 30258. https://doi.org/10.1038/s41598-026-51009-9

Landis JR, Koch GG. (1977). The measurement of observer agreement for categorical data. Biometrics 33(1): 159–174. https://pubmed.ncbi.nlm.nih.gov/843571/

Lencioni GC, de Sousa RV, de Souza Sardinha EJ, Corrêa RR, Zanella AJ. (2021). Pain assessment in horses using automatic facial expression recognition through deep learning-based modeling. PLOS One 16(10): e0258672. https://doi.org/10.1371/journal.pone.0258672

Li DL, Lee SK, Tsai YC. (2026). Deep learning-based dog expression recognition. IEEE Access 14: 5675–5683. https://doi.org/10.1109/ACCESS.2025.3650550

Li S, Zhang H, Lv Y, Wang Z, Yang H, Yu Q. (2025). Intelligent canine emotion recognition system utilizing EfficientNet-B0 architecture. 2025 IEEE 3rd International Conference on Sensors, Electronics and Computer Engineering (ICSECE), Jinzhou, China. Pp. 325–330. https://doi.org/10.1109/ICSECE65727.2025.11257013

Lundberg SM, Lee SI. (2017). A unified approach to interpreting model predictions. In: Proceedings of the 31th International Conference on Neural Information Processing Systems. Pp. 4765–4774. Curran Associates Inc., Long Beach, California, USA. https://doi.org/10.48550/arXiv.1705.07874

Martvel G, Riemer S. (2025). Automated analysis of emotional expressions in dogs based on geometric morphometrics. Scientific Reports 15: 32331. https://doi.org/10.1038/s41598-025-15741-y

Martvel G, Zamansky A, Pedretti G, Canori C, Shimshoni I, Bremhorst A. (2025). Dog facial landmarks detection and its applications for facial analysis. Scientific Reports 15: 21886. https://doi.org/10.1038/s41598-025-07040-3

Mendl M, Burman OHP, Paul ES. (2010). An integrative and functional framework for the study of animal emotion and mood. Proceedings of the Royal Society B: Biological Sciences 277(1696): 2895–2904. https://doi.org/10.1098/rspb.2010.0303

Neethirajan S. (2021). Happy cow or thinking pig? WUR Wolf—facial coding platform for measuring emotions in farm animals. AI 2(3): 342–354. https://doi.org/10.3390/ai2030021

Nie L, Li B, Du Y, Jiao F, Song X, Liu Z. (2024). Deep learning strategies with CReToNeXt-YOLOv5 for advanced pig face emotion detection. Scientific Reports 14(1): 1679. https://doi.org/10.1038/s41598-024-51755-8

Nithya R, Mariadas AEP, Anbarasu MM, Ravindran S. (2026). Intelligent NB-IoT and emotion-aware monitoring system for proactive wildlife security. Tehnički vjesnik 33(3): 1139–1147. https://doi.org/10.17559/TV-20250827002927

Ntalampiras S, Pesando Gamacchio G. (2025). Explainable classification of goat vocalizations using convolutional neural networks. PLOS One 20(4): e0318543. https://doi.org/10.1371/journal.pone.0318543

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372: n71. https://doi.org/10.1136/bmj.n71

Paul ES, Harding EJ, Mendl M. (2005). Measuring emotional processes in animals: the utility of a cognitive approach. Neuroscience & Biobehavioral Reviews 29(3): 469–491. https://doi.org/10.1016/j.neubiorev.2005.01.002

Pearline SSRA, Subbiah G, Sanu A, Charumathi Venkatesan K. (2026). Multi-QuadEmoNet: cat and dog emotion classification model from animal vocalization using multi-stage LSTM-GRU paradigm. Frontiers in Veterinary Science 13: 1799281. https://doi.org/10.3389/fvets.2026.1799281

Priya HD, Vimal S, Vignesh M, Devaguru S. (2026). Interpreting subtle facial cues in elephants: a deep learning approach to emotional state recognition. 2026 Second International Conference on Intelligent Systems for Communication, IoT and Security (ICISCoIS), Coimbatore, India. Pp. 1–6. https://doi.org/10.1109/ICISCoIS62701.2026.11447516

Saito T, Rehmsmeier M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS One 10(3): e0118432. https://doi.org/10.1371/journal.pone.0118432

Schütz AK, Krause ET, Fischer M, Müller T, Freuling CM, Conraths FJ, et al. (2022). Computer vision for detection of body posture and behavior of red foxes. Animals 12(3): 233. https://doi.org/10.3390/ani12030233

Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. (2017). Grad-CAM: visual explanations from deep networks via gradient-based localization. IEEE International Conference on Computer Vision (ICCV), Venice, Italy. Pp. 618–626. https://doi.org/10.1109/ICCV.2017.74

Shao D, He Z, Fan H, Sun K. (2023). Detection of cattle key parts based on the improved YOLOv5 algorithm. Agriculture 13(6): 1110. https://doi.org/10.3390/agriculture13061110

Sinnott RO, Aickelin U, Jia Y, Sinnott ER, Sun PY, Susanto R. (2021). Run or pat: Using deep learning to classify the species type and emotion of pets. 2021 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE), Brisbane, Australia. Pp. 1–6. https://doi.org/10.1109/CSDE53843.2021.9718465

Song H, Brady S, Ward T, Smeaton AF. (2025). A dimensional approach to canine bark analysis for assistance dog seizure signaling. arXiv preprint. https://doi.org/10.48550/arXiv.2509.18375

Subair R, Benedict S. (2024). Enhancing elephant emotion detection using YOLOv5, YOLOv8, and YOLOv9: a study on performance and explainability. 2024 IEEE Silchar Subsection Conference (SILCON 2024), Agartala, India. Pp. 1–6. https://doi.org/10.1109/SILCON63976.2024.10910543

Sui Z, Hong H, Shimtoku D, Kobayashi HH. (2024). catAction: deep learning for enhancing emotional cat–human interactions through the posture-based determination of the degrees of kittens’ defensive and offensive aggressions. International Conference on Animal-Computer Interaction, Glasgow, UK. Pp. 1–9. https://doi.org/10.1145/3702336.3702341

Sumon RI, Ali H, Akter S, Uddin SMI, Mozumder MAI, Kim HC. (2025). A deep learning-based approach for precise emotion recognition in domestic animals using EfficientNetB5 architecture. Engineering 6(1): 9. https://doi.org/10.3390/eng6010009

Suneetha C, Anitha R. (2025). Speech-based emotion recognition by using a faster region-based convolutional neural network. Multimedia Tools and Applications 84(8): 5205–5237. https://doi.org/10.1007/s11042-024-19004-2

Tanwar V. (2024). Understanding dog emotions through deep learning: A CNN-based classification framework. 2024 5th International Conference on Smart Electronics and Communication (ICOSEC), Trichy, India. Pp. 964–969. https://doi.org/10.1109/ICOSEC61587.2024.10722523

Tribethran S, Hasan NJP, Rahman A. (2024). Classification of dog emotion using transfer learning on convolutional neural network algorithm. Paradigma-Jurnal Komputer dan Informatika 26(2): 102–108. https://doi.org/10.31294/p.v26i2.5295

Vikramathithan AC, Manjunath TG, Lakshya HE, Rakshitha R. (2025). Emotion detection for cross-species using VGG16 model with Ada-Sig-Max optimizer. 2025 International Conference on Data, Energy and Communication Networks (DECoN), Bhopal, India. Pp. 1–6. https://doi.org/10.1109/DECoN67170.2025.11447459

Wang H, Shi Z, Hu R, Wang X, Chen J, Che H. (2025). Real-time fear emotion recognition in mice based on multimodal data fusion. Scientific Reports 15(1): 11797. https://doi.org/10.1038/s41598-025-95483-z

Wu Y. (2023). Emotion detection of dogs and cats using classification models and object detection model (Research project). Yeshiva University. ResearchGate. https://www.researchgate.net/publication/370729284_Emotion_Detection_of_Dogs_and_Cats_Using_Classification_Models_and_Object_Detection_Model

Xie Y. (2023). Dogs emotion recognition and parameter analysis based on EfficientNet with transfer learning. 1st International Conference on Data Analysis and Machine Learning (DAML), Kuala Lumpur, Malaysia. Pp. 240–245. https://doi.org/10.5220/0012799600003885

Yunas SU, Shahbaz A, Baxter EM, Hansen MF, Smith ML, Smith LN. (2025). Enhanced deep learning for robust stress classification in sows from facial images. Agriculture 15(15): 1675. https://doi.org/10.3390/agriculture15151675

Downloads

Published

10-11-2026

How to Cite

Mandal, B., & Kumar, D. (2026). Animal affective-state analysis using Convolutional Neural Networks: Methods, datasets, applications, challenges, and future directions: Use of CNN in animal affective-state analysis. Letters in Animal Biology, 6(1), 193–206. https://doi.org/10.62310/liab.v6i1.454

Issue

Section

Review Articles
Recieved 2026-09-25
Accepted 2026-11-06
Published 2026-11-10