Mehrabian, A. Communication without words. Psychol. Today 2, 53–56 (1968).
Ekman, P. & Friesen, W. V. Facial Action Coding System (Consulting Psychology Press, 1978).
Ekman, P. & Rosenberg, E. L. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS) (Oxford University Press, 1997).
Harashima, H., Choi, C. S. & Takebe, T. 3-d model-based synthesis of facial expressions and shape deformation. Hum. Interface 4, 157–166 (1989).
Mase, K. An application of optical flow-extraction of facial expression. In IAPR Workshop on Machine Vision and Application 195–198 (1990).
Mase, K. Recognition of facial expression from optical flow. Trans. IEICE E74(10), 3474–3483 (1991).
Matsuno, K., Lee, C. & Tsuji, S. Recognition of facial expressions using potential net and kl expansion. Trans. IEICE J77-D-I I(8), 1591–1600 (1994).
Kobayashi, H. & Hara, F. Analysis of neural network recognition characteristics of 6 basic facial expressions. In Proc. of IEEE International Workshop on Robot and Human Communication 222–227 (1994).
Filippini, C., Perpetuini, D., Cardone, D., Chiarelli, A. & Merla, A. Thermal infrared imaging-based affective computing and its application to facilitate human robot interaction: A review. Appl. Sci. 10, 2924 (2020).
Google Scholar
Goulart, C., Valadão, C., Delisle-Rodriguez, D., Caldeira, E. & Bastos, T. Emotion analysis in children through facial emissivity of infrared thermal imaging. PLoS ONE 14, e0212928 (2019).
Google Scholar
Clay-Warner, J. & Robinson, D. Infrared thermography as a measure of emotion response. Emot. Rev. 7, 157–162 (2015).
Google Scholar
Kopaczka, M., Kolk, R. & Merhof, D. A fully annotated thermal face database and its application for thermal facial expression recognition. In IEEE International Instrumentation and Measurement Technology Conference (I2MTC) 1–6 (2018).
Hammal, J., Covreur, L., Caplier, A. & Rombout, M. Facial expression classification: An approach based on the fusion of facial deformations using the transferable belief model. Int. J. Approx. Reason. 46, 542–567 (2007).
Google Scholar
Ojo, A. & Idowu, T. Improved model for facial expression classification for fear and sadness using local binary pattern histogram. J. Adv. Math. Comput. Sci. 35(5), 22–33 (2020).
Google Scholar
Kyperountas, M., Tefas, A. & Pitas, I. Salient feature and reliable classifier selection for facial expression classification. Pattern Recogn. 43, 972–986 (2010).
Google Scholar
Ali, M., Zhuang, H. & Ibrahim, K. An approach for facial expression classification. Int. J. Biom. 9, 96 (2017).
Bartlett, M., Littlewort, G. & Fasel, I. Towards social robots: Automatic evaluation of human–robot interaction by face detection and expression classification. Neural Inform. Process. Syst. (2003).
Khan, M., Khurshid, K. & Shafait, F. A spatio-spectral hybrid convolutional architecture for hyperspectral document authentication. In International Conference on Document Analysis and Recognition 1097–1102 (2019).
Rodriguez, P. et al. Deep pain: Exploiting long short-term memory networks for facial expression classification. In IEEE Transactions on Cybernetics 1–11 (2017).
Lien, J. J. J., Kanade, T., Cohn, J. F. & Li, C. C. Detection, tracking, and classification of action units in facial expression. Robot. Auton. Syst. 31(3), 131–146 (2000).
Google Scholar
Yoshitomi, Y., Miyaura, T., Tomita, S. & Kimura, S. Face identification using thermal image processing. In Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN’97 SENDAI 374–379 (1997).
Yoshitomi, Y., Miyawaki, N., Tomita, S. & Kimura, S. Facial expression recognition using thermal image processing and neural network. In Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN’97 SENDAI, Vol. 46, 542–567 (2002).
Yoshitomi, Y., Miyawaki, N., Tomita, S. & Kimura, S. Facial expression recognition using thermal image processing and neural network. In Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN’97 SENDAI, Vol. 46, 542–567 (2002).
Bijalwan, V., Balodhi, M. & Gusain, A. Human emotion recognition using thermal image processing and eigenfaces. IJESR 5(1), 34–40 (2015).
Shen, P., Wang, S. & Liu, Z. Facial expression recognition from infrared thermal videos. In Intelligent Autonomous Systems 12. Advances in Intelligent Systems and Computing, Vol. 194 (2013).
Goulart, C. et al. Visual and thermal image processing for facial specific landmark detection to infer emotions in a child–robot interaction. MDPI Sens. 19, 2844 (2019).
Google Scholar
Khan, M. M., Ingleby, M. & Ward, R. D. Automated facial expression classification and affect interpretation using infrared measurement of facial skin temperature variations. Assoc. Comput. Mach. 1, 91–113 (2006).
Prabhakaran, A., Nair, J. & Sarath, S. Thermal facial expression recognition using modified resnet152. In Advances in Computing and Network Communications 389–396 (2021).
Khan, M., Khan, M., Siddiqui, A. & Khurshid, K. An automated and efficient convolutional architecture for disguise-invariant face recognition using noise-based data augmentation and deep transfer learning. In The Visual Computer 1–15 (2021).
Bodavarapu, P. & Srinivas, P. Facial expression recognition for low resolution images using convolutional neural networks and denoising techniques. Indian J. Sci. Technol. 14, 971–983 (2021).
Google Scholar
Reddy, G., Savarni, C. & Mukherjee, S. Facial expression recognition in the wild, by fusion of deep learnt and hand-crafted features. Cogn. Syst. Res. 62, 23–34 (2020).
Google Scholar
Draw.io, v14.1.8. (accessed 10 September 2021); https://app.diagrams.net/.
Li, S. & Deng, W. Deep facial expression recognition: A survey. IEEE Trans. Affect. Comput. 1, 6535–6548 (2020).
Yu, F. & Koltun, V. Multi-scale context aggregation by dilated convolutions. In International Conference on Learning Representations (2017).
Huang, G. et al. Snapshot ensembles: Train 1, get m for free. CoRR (2017).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition 770–778 (2016).
Ioffe, S. & Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 32nd International Conference on Machine Learning, ICML (2015).
Loshchilov, I. & Hutter, F. SGDR: Stochastic gradient descent with warm restarts. ICLR (2017).
Selvaraju, R. R. et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision 618–626 (2017).
Kopaczka, M., Breuer, L., Schock, J. & Merhof, D. A modular system for detection, tracking and analysis of human faces in thermal infrared recordings. Sensors 19, 4135 (2019).
Google Scholar
Panetta, K. et al. A comprehensive database for benchmarking imaging systems. IEEE Trans. Pattern Anal. Mach. Intell. 42, 509–520. https://doi.org/10.1109/tpami.2018.2884458 (2020).
Google Scholar
Shreyas Kamath, K. M., Rajendran, R., Wan, Q., Panetta, K. & Agaian, S. S. TERNet: A deep learning approach for thermal face emotion recognition. In Mobile Multimedia/Image Processing, Security, and Applications 2019 (SPIE, 2019). https://doi.org/10.1117/12.2518708.
Samadiani, N. et al. A review on automatic facial expression recognition systems assisted by multimodal sensor data. Sensors 19, 1863 (2019).
Google Scholar
Harsih Kamar, R. J., Akash, N., Gokul, R. & Merhof, D. Facial expression recognition system using multimodal sensors. Int. J. Multidiscip. Res. Sci., Eng. Technol 1, 30–35 (2020).
Khan, M. & Curry, E. Neuro-symbolic visual reasoning for multimedia event processing: Overview, prospects and challenges. In Proceedings of the CIKM 2020 Workshops, Vol. 2699 of CEUR Workshop Proceedings (eds. Conrad, S. & Tiddi, I.) (CEUR-WS.org, 2020). http://ceur-ws.org/Vol-2699/paper10.pdf.
