Engineering Commons^™

Real-Time Facial Expression Recognition Using Edge Ai Accelerators, Mark Heath Smith

Theses and Dissertations

Facial expression recognition is a popular and challenging area of research in machine learning applications. Facial expressions are critical to human communication and allow us to convey complex thoughts and emotions beyond spoken language. The complexity of facial expressions creates a difficult problem for computer vision systems, especially edge computing systems. Current Deep Learning (DL) methods rely on large-scale Convolutional Neural Networks (CNN) which require millions of floating point operations (FLOPS) to accomplish similar image classification tasks. However, on edge and IoT devices, large-scale convolutional models can cause problems due to memory and power limitations. The intent of this work …

Ad-Corre: Adaptive Correlation-Based Loss For Facial Expression Recognition In The Wild, Ali Pourramezan Fard, Mohammad H. Mahoor

Electrical and Computer Engineering: Faculty Scholarship

Automated Facial Expression Recognition (FER) in the wild using deep neural networks is still challenging due to intra-class variations and inter-class similarities in facial images. Deep Metric Learning (DML) is among the widely used methods to deal with these issues by improving the discriminative power of the learned embedded features. This paper proposes an Adaptive Correlation (Ad-Corre) Loss to guide the network towards generating embedded feature vectors with high correlation for within-class samples and less correlation for between-class samples. Ad-Corre consists of 3 components called Feature Discriminator, Mean Discriminator, and Embedding Discriminator. We design the Feature Discriminator component to guide …

Facial Landmark Feature Fusion In Transfer Learning Of Child Facial Expressions, Megan A. Witherow, Manar D. Samad, Norou Diawara, Khan M. Iftekharuddin

Electrical & Computer Engineering Faculty Publications

Automatic classification of child facial expressions is challenging due to the scarcity of image samples with annotations. Transfer learning of deep convolutional neural networks (CNNs), pretrained on adult facial expressions, can be effectively finetuned for child facial expression classification using limited facial images of children. Recent work inspired by facial age estimation and age-invariant face recognition proposes a fusion of facial landmark features with deep representation learning to augment facial expression classification performance. We hypothesize that deep transfer learning of child facial expressions may also benefit from fusing facial landmark features. Our proposed model architecture integrates two input branches: a …

Facial Expression Recognition With Independent Subspace Analysis Based Feature Learning, Yongjie Zhan, Long Fei, Yikun Bu

Journal of System Simulation

Abstract: Hand-designed features (such as Gabor, LBP) has been widely employed in facial expression recognition. In the real-world applications of facial expression recognition, it is very difficult to achieve perfect face alignment because of the impact of complex background and the limitations of face alignment approaches. Independent Subspace Analysis (ISA) is an unsupervised feature learning method, which can be used to learn phase-invariant visual features from images. The problem of facial expression recognition based on ISA in the situation of not precise face alignment was investigated. Through analyzing the facial expression recognition performances with different subspace size, it was turned …

Deep Siamese Neural Networks For Facial Expression Recognition In The Wild, Wassan Hayale

Electronic Theses and Dissertations

The variation of facial images in the wild conditions due to head pose, face illumination, and occlusion can significantly affect the Facial Expression Recognition (FER) performance. Moreover, between subject variation introduced by age, gender, ethnic backgrounds, and identity can also influence the FER performance. This Ph.D. dissertation presents a novel algorithm for end-to-end facial expression recognition, valence and arousal estimation, and visual object matching based on deep Siamese Neural Networks to handle the extreme variation that exists in a facial dataset. In our main Siamese Neural Networks for facial expression recognition, the first network represents the classification framework, where we …

Automated Recognition Of Facial Affect Using Deep Neural Networks, Behzad Hasani

Electronic Theses and Dissertations

Automated Facial Expression Recognition (FER) has been a topic of study in the field of computer vision and machine learning for decades. In spite of efforts made to improve the accuracy of FER systems, existing methods still are not generalizable and accurate enough for use in real-world applications. Many of the traditional methods use hand-crafted (a.k.a. engineered) features for representation of facial images. However, these methods often require rigorous hyper-parameter tuning to achieve favorable results.

Recently, Deep Neural Networks (DNNs) have shown to outperform traditional methods in visual object recognition. DNNs require huge data as well as powerful computing units …

Facial Action Unit Detection With Deep Convolutional Neural Networks, Siddhesh Padwal

Electronic Theses and Dissertations

The facial features are the most important tool to understand an individual's state of mind. Automated recognition of facial expressions and particularly Facial Action Units defined by Facial Action Coding System (FACS) is challenging research problem in the field of computer vision and machine learning. Researchers are working on deep learning algorithms to improve state of the art in the area. Automated recognition of facial action units has man applications ranging from developmental psychology to human robot interface design where companies are using this technology to improve their consumer devices (like unlocking phone) and for entertainment like FaceApp. Recent studies …

A Computational Study On Aging Effect For Facial Expression Recognition, Elena Sönmez

Turkish Journal of Electrical Engineering and Computer Sciences

This work uses newly introduced variations of the sparse representation-based classifier (SRC) to challenge the issue of automatic facial expression recognition (FER) with faces belonging to a wide span of ages. Since facial expression is one of the most powerful and immediate ways to disclose individuals? emotions and intentions, the study of emotional traits is an active research topic both in psychology and in engineering fields. To date, automatic FER systems work well with frontal and clean faces, but disturbance factors can dramatically decrease their performance. Aging is a critical disruption element, which is present in any real-world situation and …

Local Directional-Structural Pattern For Person-Independent Facial Expression Recognition, Farkhod Makhmudkhujaev, Md Tauhid Bin Iqbal, Byungyong Ryu, Oksam Chae

Turkish Journal of Electrical Engineering and Computer Sciences

Existing popular descriptors for facial expression recognition often suffer from inconsistent feature description, experiencing poor accuracies. We present a new local descriptor, local directional-structural pattern (LDSP), in this work to address this issue. Unlike the existing local descriptors using only the texture or edge information to represent the local structure of a pixel, the proposed LDSP utilizes the positional relationship of the top edge responses of the target pixel to extract more detailed structural information of the local texture. We further exploit such information to characterize expression-affiliated crucial textures while discarding the random noisy patterns. Moreover, we introduce a globally …

Studying Facial Expression Recognition And Imitation Ability Of Children With Autism Spectrum Disorder In Interaction With A Social Robot, Farzaneh Askari

Electronic Theses and Dissertations

Children with Autism Spectrum Disorder (ASD) experience limited abilities in recognizing non-verbal elements of social interactions such as facial expressions [1]. They also show deficiencies in imitating facial expressions in social situations. In this Master thesis, we focus on studying the ability of children with ASD in recognizing facial expressions and imitating the expressions using a rear-projected expressive humanoid robot, called Ryan. Recent studies show that social robots such as Ryan have great potential for autism therapy. We designed and developed three studies, first to evaluate the ability of children with ASD in recognizing facial expressions that are presented to …

Compact Local Gabor Directional Number Pattern For Facial Expression Recognition, Zhengyan Zhang, Guanming Lu, Jingjie Yan, Haibo Li, Ning Sun, Xia Li

Turkish Journal of Electrical Engineering and Computer Sciences

This paper explores a novel method to represent face images for facial expression recognition; it is named compact local Gabor directional number pattern (CLGDNP). By convolving the face images with Gabor filters, we encode the magnitude and phase response images in each scale, and calculate the histograms in several nonoverlapping regions of each encoded image. Finally, we obtain two spatial histogram sequences by the aid of the mean pooling technology and concatenate them to form the facial descriptor. Moreover, for evaluating the performance of the proposed method, we employ a support vector machine to conduct some extensive classification experiments on …

Adaptive Joint Block-Weighted Collaborative Representation For Facial Expression Recognition, Zhe Sun, Zhengping Hu, Meng Wang, Shuhuan Zhao

Turkish Journal of Electrical Engineering and Computer Sciences

Facial expression recognition (FER) plays a significant role in human-computer interactions. Recently, regularized linear representation-based classification has achieved satisfying results in FER. Considering that different blocks in a sample should contribute differently to the representation and classification, we propose an adaptive joint block-weighted collaborative representation-based classification (JBW_CRC) method to effectively exploit the similarity and distinctiveness of different blocks. In JBW_CRC, samples are divided into different blocks and each block of the query sample is represented as a feature vector. Each feature vector is coded on its related block dictionary, which considers the similarity among the feature vectors. Additionally, the distinctiveness …

A Facial Component-Based System For Emotion Classification, Elena Sönmez, Songül Albayrak

Turkish Journal of Electrical Engineering and Computer Sciences

Smart environments with ubiquitous computers are the next generation of information technology, which requires improved human--computer interfaces. That is, the computer of the future must be aware of the people in its environment; it must know their identities and must understand their moods. Despite the great effort made in the past decades, the development of a system capable of automatic facial emotion recognition is still rather difficult. In this paper, we challenge the benchmark algorithm on emotion classification of the Extended Cohn-Kanade (CK$+)$ database, and we present a facial component-based system for emotion classification, which beats the given benchmark performance: …