Physical Sciences and Mathematics Commons^™

An Ml Based Digital Forensics Software For Triage Analysis Through Face Recognition, Gaurav Gogia, Parag H. Rughani

Journal of Digital Forensics, Security and Law

Since the past few years, the complexity and heterogeneity of digital crimes has increased exponentially, which has made the digital evidence & digital forensics paramount for both criminal investigation and civil litigation cases. Some of the routine digital forensic analysis tasks are cumbersome and can increase the number of pending cases especially when there is a shortage of domain experts. While the work is not very complex, the sheer scale can be taxing. With the current scenarios and future predictions, crimes are only going to become more complex and the precedent of collecting and examining digital evidence is only going …

The Model 2.0 And Friends: An Interim Report, Garrison W. Cottrell, Martha Gahl, Shubham Kulkarni, Shashank Venkatramani, Yash Shah, Keyu Long, Xuzhe Zhi, Shivaank Agarwal, Cody Li, Jingyuan He, Thomas Fischer

MODVIS Workshop

Last year, I reported on preliminary results of an anatomically-inspired deep learning model of the visual system and its role in explaining the face inversion effect. This year, I will report on new results and some variations on network architectures that we have explored, mainly as a way to generate discussion and get feedback. This is by no means a polished, final presentation!

We look forward to the group’s suggestions for these projects.

Computational Intelligence And Soft Computing Paradigm For Cheating Detection In Online Examinations, Sanaa Kaddoura, Shweta Vincent, D. Jude Hemanth

All Works

Covid-19 has been a life-changer in the sphere of online education. With complete lockdown in various countries, there has been a tumultuous increase in the need for providing online education, and hence, it has become mandatory for examiners to ensure that a fair methodology is followed for evaluation, and academic integrity is met. A plethora of literature is available related to methods to mitigate cheating during online examinations. A systematic literature review (SLR) has been followed in our article which aims at introducing the research gap in terms of the usage of soft computing techniques to combat cheating during online …

Disagreement Matters: Exploring Internal Diversification For Redundant Attention In Generic Facial Action Analysis, Xiaotian Li, Zheng Zhang, Xiang Zhang, Taoyue Wang, Zhihua Li, Huiyuan Yang, Umur Ciftci, Qiang Ji, Jeffrey Cohn, Lijun Yin

Computer Science Faculty Research & Creative Works

This paper demonstrates the effectiveness of a diversification mechanism for building a more robust multi-attention system in generic facial action analysis. While previous multi-attention (e.g., visual attention and self-attention) research on facial expression recognition (FER) and Action Unit (AU) detection have been thoroughly studied to focus on "external attention diversification", where attention branches localize different facial areas, we delve into the realm of "internal attention diversification" and explore the impact of diverse attention patterns within the same Region of Interest (RoI). Our experiments reveal that variability in attention patterns significantly impacts model performance, indicating that unconstrained multi-attention plagued by redundancy …

Maximum Spatial Perturbation Consistency For Unpaired Image-To-Image Translation, Yanwu Xu, Shaoan Xie, Wenhao Wu, Kun Zhang, Mingming Gong, Kayhan Batmanghelich

Machine Learning Faculty Publications

Unpaired image-to-image translation (I2I) is an ill-posed problem, as an infinite number of translation functions can map the source domain distribution to the target distribution. Therefore, much effort has been put into designing suitable constraints, e.g., cycle consistency (CycleGAN), geometry consistency (GCGAN), and contrastive learning-based constraints (CUTGAN), that help better pose the problem. However, these well-known constraints have limitations: (1) they are either too restrictive or too weak for specific I2I tasks; (2) these methods result in content distortion when there is a significant spatial variation between the source and target domains. This paper proposes a universal regularization technique called …

High-Resolution Face Swapping Via Latent Semantics Disentanglement, Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, Shengfeng He

Research Collection School Of Computing and Information Systems

We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure at-tributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending …

Measuring The Relationship Of Gender Misclassification And Automated Face Recognition Match Accuracy Relative To Skin Tone, Afi Edem-Edi Gbekevi

Theses and Dissertations

The gap of accuracy observed in some commercial face analytic systems based on race and gender raised questions about the equity and fairness of those systems. Since these systems are part of several applications today, some more critical than others, it urges designers to detect and mitigate any sources of bias. In this thesis, we begin by clarifying the confusion between face analytic, face recognition, and face processing systems. Then, we analyze gender classification accuracy using two datasets and three classifiers. The Pilot Parliaments Benchmark dataset is examined with an open-source algorithm to corroborate the gender shade. Secondly, the Morph …

Generating Face Images With Attributes For Free, Yaoyao Liu, Qianru Sun, He Xiangnan, Liu An-An, Su Yuting, Chua Tat-Seng

Research Collection School Of Computing and Information Systems

With superhuman-level performance of face recognition, we are more concerned about the recognition of fine-grained attributes, such as emotion, age, and gender. However, given that the label space is extremely large and follows a long-tail distribution, it is quite expensive to collect sufficient samples for fine-grained attributes. This results in imbalanced training samples and inferior attribute recognition models. To this end, we propose the use of arbitrary attribute combinations, without human effort, to synthesize face images. In particular, to bridge the semantic gap between high-level attribute label space and low-level face image, we propose a novel neural-network-based approach that maps …

An Evolutionary-Based Image Classification Approach Through Facial Attributes, Seli̇m Yilmaz, Cemi̇l Zalluhoğlu

Turkish Journal of Electrical Engineering and Computer Sciences

With the recent developments in technology, there has been a significant increase in the studies on analysisof human faces. Through automatic analysis of faces, it is possible to know the gender, emotional state, and even theidentity of people from an image. Of them, identity or face recognition has became the most important task whichhas been studied for a long time now as it is crucial to take measurements for public security, credit card verification,criminal identification, and the like. In this study, we have proposed an evolutionary-based framework that relies ongenetic programming algorithm to evolve a binary- and multilabel image classifier …

Presentation Attack Detection For Face Recognition Using Remotephotoplethysmography And Cascaded Fusion, Mehmet Fati̇h Gündoğar, Çi̇ğdem Eroğlu Erdem

Turkish Journal of Electrical Engineering and Computer Sciences

Spoofing (presentation) attacks are important threats for face recognition and authentication systems, which try to deceive them by presenting an image or video of a different subject, or by using a 3D mask. Remote (non-contact) photoplethysmography (rPPG) is useful for liveness detection using a facial video by estimating the heart-rate of the subject. In this paper, we first compare the presentation attack detection performance of three different rPPG-based heart rate estimation methods on four datasets (3DMAD, Replay-Attack, Replay-Mobile, and MSU-MFSD). We also present a cascaded fusion system, which utilizes a multistage ensemble of classifiers using rPPG, motion-based (including head-pose, eye-gaze …

Single And Differential Morph Attack Detection, Baaria Chaudhary

Graduate Theses, Dissertations, and Problem Reports

Face recognition systems operate on the assumption that a person's face serves as the unique link to their identity. In this thesis, we explore the problem of morph attacks, which have become a viable threat to face verification scenarios precisely because of their inherent ability to break this unique link. A morph attack occurs when two people who share similar facial features morph their faces together such that the resulting face image is recognized as either of two contributing individuals. Morphs inherit enough visual features from both individuals that both humans and automatic algorithms confuse them. The contributions of this …

The Nearest Polyhedral Convex Conic Regions For High-Dimensional Classification, Hakan Çevi̇kalp, Emre Çi̇men, Gürkan Öztürk

Turkish Journal of Electrical Engineering and Computer Sciences

In the nearest-convex-model type classifiers, each class in the training set is approximated with a convexclass model, and a test sample is assigned to a class based on the shortest distance from the test sample to these classmodels. In this paper, we propose new methods for approximating the distances from test samples to the convex regionsspanned by training samples of classes. To this end, we approximate each class region with a polyhedral convex conicregion by utilizing polyhedral conic functions (PCFs) and its extension, extended PCFs. Then, we derive the necessary formulations for computing the distances from test samples to these …

A User Study Of A Wearable System To Enhance Bystanders’ Facial Privacy, Alfredo J. Perez, Sherali Zeadally, Scott Griffith, Luis Y. Matos Garcia, Jaouad A. Mouloud

Information Science Faculty Publications

The privacy of users and information are becoming increasingly important with the growth and pervasive use of mobile devices such as wearables, mobile phones, drones, and Internet of Things (IoT) devices. Today many of these mobile devices are equipped with cameras which enable users to take pictures and record videos anytime they need to do so. In many such cases, bystanders’ privacy is not a concern, and as a result, audio and video of bystanders are often captured without their consent. We present results from a user study in which 21 participants were asked to use a wearable system called …

A User Study Of A Wearable System To Enhance Bystanders’ Facial Privacy, Alfredo J. Perez, Sherali Zeadally, Scott Griffith, Luis Y. Matos Garcia, Jason A. Mouloud

Computer Science Faculty Publications

The privacy of users and information are becoming increasingly important with the growth and pervasive use of mobile devices such as wearables, mobile phones, drones, and Internet of Things (IoT) devices. Today many of these mobile devices are equipped with cameras which enable users to take pictures and record videos anytime they need to do so. In many such cases, bystanders’ privacy is not a concern, and as a result, audio and video of bystanders are often captured without their consent. We present results from a user study in which 21 participants were asked to use a wearable system called …

Amora: Black-Box Adversarial Morphing Attack, Run Wang, Felix Juefei-Xu, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Yang Liu

Research Collection School Of Computing and Information Systems

Nowadays, digital facial content manipulation has become ubiquitous and realistic with the success of generative adversarial networks (GANs), making face recognition (FR) systems suffer from unprecedented security concerns. In this paper, we investigate and introduce a new type of adversarial attack to evade FR systems by manipulating facial content, called adversarial morphing attack (a.k.a. Amora). In contrast to adversarial noise attack that perturbs pixel intensity values by adding human-imperceptible noise, our proposed adversarial morphing attack works at the semantic level that perturbs pixels spatially in a coherent manner. To tackle the black-box attack problem, we devise a simple yet effective …

Embedded Face Recognition Combined Mutual Information With Log-Gabor Feature, Jihua Ye, Qingping Lan, Changhong Liu, Shimin Wang

Journal of System Simulation

Abstract: Log-Gabor functions have extended tails at the high frequency end, can effectively improve the shortcoming of ordinary Gabor functions which would over-represent the low frequency components and under-represent the high frequency components, and Log-Gabor filter has no DC components, the bandwidth is not limited, so Log-Gabor filter is more suitable than Gabor filter to extract the face feature. After extracting features of a face image by a Log-Gabor filter bank which is composition of four scales and six orientations, the amount of data is 24 times of the original, but the embedded equipment resources are limited, it is difficult …

Face Recognition Algorithm Based On Mixed Multiple Distance Image And Linear Discriminant Analysis, Yaling Cheng, Aiping Tan, Zhang Min

Journal of System Simulation

Abstract: The existing face recognition algorithms used in intelligent monitoring system are mainly applied to short distance images, which still have the problem of low face recognition rate when being applied to long distance images because the image quality decreases as the distance grows. To improve the face recognition accuracy in long distance images, a novel face recognition algorithm was proposed based on using mixed multiple different distance images and Linear Discriminant Analysis (LDA). The proposed algorithm used mixed images extracted from multiple different distances to train images, and used bilinear interpolation method to normalize the image set, and then …

Sparse Representation Based Private Face Recognition In The Cloud, Liu Yan, Jin Xin, Zhao Geng, Xiaodong Li, Yingya Chen, Guo Kui

Journal of System Simulation

Abstract: In order to protect user privacy, identification computing of user facial image data in the cloud was accomplished, the sparse representation based private face recognition method in the cloud was proposed. Terminal users collected face images and compared with face images in cloud database and then determined whether faces the terminal acquired belonged to the cloud face database, but both could not achieve each other's face image content. The method processed face images in database and cloud into sparse representation via a third-party terminal, then using Paillier and oblivious transfer homomorphic encryption algorithm to contrast sparse representation coefficient vector …

Improving Visual Recognition With Unlabeled Data, Aruni Roy Chowdhury

Doctoral Dissertations

The success of deep neural networks has resulted in computer vision systems that obtain high accuracy on a wide variety of tasks such as image classification, object detection, semantic segmentation, etc. However, most state-of-the-art vision systems are dependent upon large amounts of labeled training data, which is not a scalable solution in the long run. This work focuses on improving existing models for visual object recognition and detection without being dependent on such large-scale human-annotated data. We first show how large numbers of hard examples (cases where an existing model makes a mistake) can be obtained automatically from unlabeled video …

Face Recognition Method Based On Cost-Sensitive Supervised Manifold Learning, Yeqin Cui, Jianguo Gao

Journal of System Simulation

Abstract: Existing subspace learning-based face recognition methods assume the same loss from all misclassifications. In the real-world face recognition applications, however, different misclassifications can lead to different losses. Motivated by this concern, a cost-sensitive supervised manifold learning approach for face recognition was proposed. The proposed approach incorporated a cost matrix to specify the different costs associated with misclassifications of subjects, into locality preserving projection algorithm, which devised the corresponding cost-sensitive methods, namely, cost-sensitive locality preserving projections (Cos-Sen LPP), to achieve a minimal overall loss. Three face databases were put into the experiments and experimental results show that Cos-Sen LPP method …

Research On Face Recognition Method Under Uncontrolled Illumination Variation, Kong Rui, Zhang Bing

Journal of System Simulation

Abstract: A new face recognition algorithm is proposed with high recognition rate under uncontrolled illumination conditions. The new algorithm process face images in advance using Weber local descriptor, which means that the processed image is insensitive to illumination changing. An improved linear discriminant analysis algorithm is adopt for feature extracting, finally, nearest neighbor classifier based on Euclidean distance is applied to classify. The new algorithm is tested on Yale and The Extended Yale Database B face database respectively, in comparison with classic face recognition algorithms, the performance of the proposed method is superior to other's under uncontrolled illumination variation …

Color Face Image Recognition Based On Lbpt Method, Jihua Ye, Yahui Chen, Shimin Wang

Journal of System Simulation

Abstract: Aiming to the shortcomings of exiting algorithm to obtain better color face image information for color facial image recognition, the LBPT algorithm was proposed to realize the high efficiency recognition of color face image. LBPT algorithm reflected the texture features of gray image through adaptively obtaining neighborhood radius, ascertaining the relationship between neighborhood radius and neighborhood pixel number, setting threshold. The RGB color model was used to separate the color face image into the R,G,B three component diagrams. The LBPT algorithm was used to obtain the feature of the component diagrams. In order to realize further recognition, the method …

Choosing The Structure Of Convolutional Neural Networks For Face Recognition, Kabul Khudaybergenov

Bulletin of National University of Uzbekistan: Mathematics and Natural Sciences

Evaluating the number of hidden neurons and hidden layers necessary for solving of face recognition, pattern recognition and classification tasks is one of the key problems in artificial neural networks. In this note, we show that artificial neural network with a two hidden layer feed forward neural network with d inputs, d neurons in the first hidden layer, 2d+2 neurons in the second hidden layer, k outputs and with a sigmoidal infinitely differentiable function can solve face recognition tasks. This result can be applied to design pattern recognition and classification models with optimal structure in the number of hidden neurons …

An Empirical Study On Correlation Between Coverage And Robustness For Deep Neural Networks, Yizhen Dong, Peixin Zhang, Jingyi Wang, Shuang Liu, Jun Sun, Jianye Hao, Xinyu Wang, Li Wang, Jinsong Dong, Ting Dai

Research Collection School Of Computing and Information Systems

Deep neural networks (DNN) are increasingly applied in safety-critical systems, e.g., for face recognition, autonomous car control and malware detection. It is also shown that DNNs are subject to attacks such as adversarial perturbation and thus must be properly tested. Many coverage criteria for DNN since have been proposed, inspired by the success of code coverage criteria for software programs. The expectation is that if a DNN is well tested (and retrained) according to such coverage criteria, it is more likely to be robust. In this work, we conduct an empirical study to evaluate the relationship between coverage, robustness and …

Lightweight Privacy-Preserving Ensemble Classification For Face Recognition, Zhuo Ma, Yang Liu, Ximeng Liu, Jianfeng Ma, Kui Ren

Research Collection School Of Computing and Information Systems

The development of machine learning technology and visual sensors is promoting the wider applications of face recognition into our daily life. However, if the face features in the servers are abused by the adversary, our privacy and wealth can be faced with great threat. Many security experts have pointed out that, by 3-D-printing technology, the adversary can utilize the leaked face feature data to masquerade others and break the E-bank accounts. Therefore, in this paper, we propose a lightweight privacy-preserving adaptive boosting (AdaBoost) classification framework for face recognition (POR) based on the additive secret sharing and edge computing. First, we …

Evaluation And Understandability Of Face Image Quality Assessment, Mohammad I. Nouyed

Graduate Theses, Dissertations, and Problem Reports

Face image quality assessment (FIQA) has been an area of interest to researchers as a way to improve the face recognition accuracy. By filtering out the low quality images we can reduce various difficulties faced in unconstrained face recognition, such as, failure in face or facial landmark detection or low presence of useful facial information. In last decade or so, researchers have proposed different methods to assess the face image quality, spanning from fusion of quality measures to using learning based methods. Different approaches have their own strength and weaknesses. But, it is hard to perform a comparative assessment of …

Facepet: Enhancing Bystanders' Facial Privacy With Smart Wearables/Internet Of Things, Alfredo J. Perez, Sherali Zeadally, Luis Y. Matos Garcia, Jaouad A. Mouloud, Scott Griffith

Information Science Faculty Publications

Given the availability of cameras in mobile phones, drones and Internet-connected devices, facial privacy has become an area of major interest in the last few years, especially when photos are captured and can be used to identify bystanders’ faces who may have not given consent for these photos to be taken and be identified. Some solutions to protect facial privacy in photos currently exist. However, many of these solutions do not give a choice to bystanders because they rely on algorithms that de-identify photos or protocols to deactivate devices and systems not controlled by bystanders, thereby being dependent on the …

Facepet: Enhancing Bystanders’ Facial Privacy With Smart Wearables/Internet Of Things, Alfredo J. Perez, Sherali Zeadally, Luis Y. Matos Garcia, Jason A. Mouloud, Scott Griffith

Computer Science Faculty Publications

Given the availability of cameras in mobile phones, drones and Internet-connected devices, facial privacy has become an area of major interest in the last few years, especially when photos are captured and can be used to identify bystanders’ faces who may have not given consent for these photos to be taken and be identified. Some solutions to protect facial privacy in photos currently exist. However, many of these solutions do not give a choice to bystanders because they rely on algorithms that de-identify photos or protocols to deactivate devices and systems not controlled by bystanders, thereby being dependent on the …

Development Of Portable Embedded Face Recognition Prototype, Prasad S. Garapati

Theses and Dissertations

In this thesis, we focus on developing a prototype for collecting multispectral images for face recognition. With such application, the quality of the image is an important factor that affects the accuracy of the recognition. However, the sensory data either provided in RGB channel or in NIR channel. The images acquired in RGB and NIR were never aligned in previous developments during my literature study. Thus, I propose an embedded system prototype capable of acquiring NIR and RGB images providing multispectral information which is computation efficient for real time face recognition applications.

Covariance Pooling For Facial Expression Recognition, D. Acharya, Zhiwu Huang, D. Paudel, Gool L. Van

Research Collection School Of Computing and Information Systems

Classifying facial expressions into different categories requires capturing regional distortions of facial landmarks. We believe that second-order statistics such as covariance is better able to capture such distortions in regional facial features. In this work, we explore the benefits of using a manifold network structure for covariance pooling to improve facial expression recognition. In particular, we first employ such kind of manifold networks in conjunction with traditional convolutional networks for spatial pooling within individual image feature maps in an end-to-end deep learning manner. By doing so, we are able to achieve a recognition accuracy of 58.14% on the validation set …