Engineering Commons^™

Person Identification From Streaming Surveillance Video Using Mid-Level Features From Joint Action-Pose Distribution, Binu M. Nair, Vijayan K. Asari

Vijayan K. Asari

We propose a real time person identification algorithm for surveillance based scenarios from low-resolution streaming video, based on mid-level features extracted from the joint distribution of various types of human actions and human poses. The proposed algorithm uses the combination of an auto-encoder based action association framework which produces per-frame probability estimates of the action being performed, and a pose recognition framework which gives per-frame body part locations. The main focus in this manuscript is to effectively combine these per-frame action probability estimates and pose trajectories from a short temporal window to obtain mid-level features. We demonstrate that these mid-level …

Volume Component Analysis For Classification Of Lidar Data, Nina M. Varney, Vijayan K. Asari

Vijayan K. Asari

One of the most difficult challenges of working with LiDAR data is the large amount of data points that are produced. Analysing these large data sets is an extremely time consuming process. For this reason, automatic perception of LiDAR scenes is a growing area of research. Currently, most LiDAR feature extraction relies on geometrical features specific to the point cloud of interest. These geometrical features are scene-specific, and often rely on the scale and orientation of the object for classification. This paper proposes a robust method for reduced dimensionality feature extraction of 3D objects using a volume component analysis (VCA) …

Scene Projection By Non-Linear Transforms To A Geo-Referenced Map For Situational Awareness, Kevin C. Krucki, Vijayan K. Asari

Vijayan K. Asari

There are many transportation and surveillance cameras currently in use in major cities that are close to the ground and show scenes from a perspective point of view. It can be difficult to follow an object of interest across multiple cameras if many of these cameras are in the same area due to the different orientations of these cameras. This is especially true when compared to wide area aerial surveillance (WAAS). To correct this problem, this research provides a method to non-linearly transform current camera perspective views into real world coordinates that can be placed on a map. Using a …

Robust Textural Features For Real Time Face Recognition, Chen Cui, Vijayan K. Asari, Andrew D. Braun

Vijayan K. Asari

Automatic face recognition in real life environment is challenged by various issues such as the object motion, lighting conditions, poses and expressions. In this paper, we present the development of a system based on a refined Enhanced Local Binary Pattern (ELBP) feature set and a Support Vector Machine (SVM) classifier to perform face recognition in a real life environment. Instead of counting the number of 1's in ELBP, we use the 8-bit code of the thresholded data as per the ELBP rule, and then binarize the image with a predefined threshold value, removing the small connections on the binarized image. …

State Preserving Extreme Learning Machine For Face Recognition, Md. Zahangir Alom, Paheding Sidike, Vijayan K. Asari, Tarek M. Taha

Vijayan K. Asari

Extreme Learning Machine (ELM) has been introduced as a new algorithm for training single hidden layer feed-forward neural networks (SLFNs) instead of the classical gradient-based algorithms. Based on the consistency property of data, which enforce similar samples to share similar properties, ELM is a biologically inspired learning algorithm with SLFNs that learns much faster with good generalization and performs well in classification applications. However, the random generation of the weight matrix in current ELM based techniques leads to the possibility of unstable outputs in the learning and testing phases. Therefore, we present a novel approach for computing the weight matrix …

Video-To-Video Pose And Expression Invariant Face Recognition Using Volumetric Directional Pattern, Vijayan K. Asari, Almabrok Essa

Vijayan K. Asari

Face recognition in video has attracted attention as a cryptic method of human identification in surveillance systems. In this paper, we propose an end-to-end video face recognition system, addressing a difficult problem of identifying human faces in video due to the presence of large variations in facial pose and expression, and poor video resolution. The proposed descriptor, named Volumetric Directional Pattern (VDP), is an oriented and multi-scale volumetric descriptor that is able to extract and fuse the information of multi frames, temporal (dynamic) information, and multiple poses and expressions of faces in input video to produce feature vectors, which are …

Recent Progress In Wide-Area Surveillance: Protecting Our Pipeline Infrastructure, Vijayan K. Asari, Paheding Sidike, Chen Cui, Varun Santhaseelan

Vijayan K. Asari

The pipeline industry has millions of miles of pipes buried along the length and breadth of the country. Since none of the areas through which pipelines run are to be used for other activities, it needs to be monitored so as to know whether the right-of-way (RoW) of the pipeline is encroached upon at any point in time. Rapid advances made in the area of sensor technology have enabled the use of high end video acquisition systems to monitor the RoW of pipelines. The images captured by aerial data acquisition systems are affected by a host of factors that include …

Multiple Object Detection In Hyperspectral Imagery Using Spectral Fringe-Adjusted Joint Transform Correlator, Paheding Sidike, Vijayan K. Asari, Mohammad S. Alam

Vijayan K. Asari

Hyperspectral imaging (HSI) sensors provide plenty of spectral information to uniquely identify materials by their reflectance spectra, and this information has been effectively used for object detection and identification applications. Joint transform correlation (JTC) based object detection techniques in HSI have been proposed in the literatures, such as spectral fringe-adjusted joint transform correlation (SFJTC) and with its several improvements. However, to our knowledge, the SFJTC based techniques were designed to detect only similar patterns in hyperspectral data cube and not for dissimilar patterns. Thus, in this paper, a new deterministic object detection approach using SFJTC is proposed to perform multiple …

Efficient Thermal Image Segmentation Through Integration Of Nonlinear Enhancement With Unsupervised Active Contour Model, Fatema Albalooshi, Evan Krieger, Paheding Sidike, Vijayan K. Asari

Vijayan K. Asari

Thermal images are exploited in many areas of pattern recognition applications. Infrared thermal image segmentation can be used for object detection by extracting regions of abnormal temperatures. However, the lack of texture and color information, low signal-to-noise ratio, and blurring effect of thermal images make segmenting infrared heat patterns a challenging task. Furthermore, many segmentation methods that are used in visible imagery may not be suitable for segmenting thermal imagery mainly due to their dissimilar intensity distributions. Thus, a new method is proposed to improve the performance of image segmentation in thermal imagery. The proposed scheme efficiently utilizes nonlinear intensity …

Intrusion Detection In Aerial Imagery For Protecting Pipeline Infrastructure, Paheding Sidike, Almabrok Essa, Vijayan K. Asari

Vijayan K. Asari

We present an automated mechanism that can detect and issue warnings of machinery threat such as the presence of construction vehicles on pipeline right-of-way. The proposed scheme models the human visual perception concepts to extract fine details of objects by utilizing the corners and gradient histogram information in pyramid levels. Two real-world aerial image datasets are used for testing and evaluation.

Dense Point-Cloud Representation Of A Scene Using Monocular Vision, Yakov Diskin, Vijayan K. Asari

Vijayan K. Asari

We present a three-dimensional (3-D) reconstruction system designed to support various autonomous navigation applications. The system presented focuses on the 3-D reconstruction of a scene using only a single moving camera. Utilizing video frames captured at different points in time allows us to determine the depths of a scene. In this way, the system can be used to construct a point-cloud model of its unknown surroundings. We present the step-by-step methodology and analysis used in developing the 3-D reconstruction technique. We present a reconstruction framework that generates a primitive point cloud, which is computed based on feature matching and depth …

Gaussian Weighted Neighborhood Connectivity Of Nonlinear Line Attractor For Learning Complex Manifolds, Theus H. Aspiras, Vijayan K. Asari, Wesam Sakla

Vijayan K. Asari

The human brain has the capability to process high quantities of data quickly for detection and recognition tasks. These tasks are made simpler by the understanding of data, which intentionally removes redundancies found in higher dimensional data and maps the data onto a lower dimensional space. The brain then encodes manifolds created in these spaces, which reveal a specific state of the system. We propose to use a recurrent neural network, the nonlinear line attractor (NLA) network, for the encoding of these manifolds as specific states, which will draw untrained data towards one of the specific states that the NLA …

Histogram Of Oriented Phase And Gradient (Hopg) Descriptor For Improved Pedestrian Detection, Hussin Ragb, Vijayan K. Asari

Vijayan K. Asari

This paper presents a new pedestrian detection descriptor named Histogram of Oriented Phase and Gradient (HOPG) based on a combination of the Histogram of Oriented Phase (HOP) features and the Histogram of Oriented Gradient features (HOG). The proposed descriptor extracts the image information using both the gradient and phase congruency concepts. Although the HOG based method has been widely used in the human detection systems, it lacks to deal effectively with the images impacted by the illumination variations and cluttered background. By fusing HOP and HOG features, more structural information can be identified and localized in order to obtain more …

Multiclass Object Detection With Single Query In Hyperspectral Imagery Using Class-Associative Spectral Fringe-Adjusted Joint Transform Correlation, Paheding Sidike, Vijayan K. Asari, Mohammad S. Alam

Vijayan K. Asari

We present a deterministic object detection algorithm capable of detecting multiclass objects in hyperspectral imagery (HSI) without any training or preprocessing. The proposed method, which is named class-associative spectral fringe-adjusted joint transform correlation (CSFJTC), is based on joint transform correlation (JTC) between object and nonobject spectral signatures to search for a similar match, which only requires one query (training-free) from the object's spectral signature. Our method utilizes class-associative filtering, modified Fourier plane image subtraction, and fringe-adjusted JTC techniques in spectral correlation domain to perform the object detection task. The output of CSFJTC yields a pair of sharp correlation peaks for …

Directional Ringlet Intensity Feature Transform For Tracking, Evan Krieger, Paheding Sidike, Theus H. Aspiras, Vijayan K. Asari

Vijayan K. Asari

The challenges existing for current intensity-based histogram feature tracking methods in wide area motion imagery include object structural information distortions and background variations, such as different pavement or ground types. All of these challenges need to be met in order to have a robust object tracker, while attaining to be computed at an appropriate speed for real-time processing. To achieve this we propose a novel method, Directional Ringlet Intensity Feature Transform (DRIFT), that employs Kirsch kernel filtering and Gaussian ringlet feature mapping. We evaluated the DRIFT on two challenging datasets, namely Columbus Large Image Format (CLIF) and Large Area Image …

Histogram Of Oriented Phase (Hop): A New Descriptor Based On Phase Congruency, Hussin Ragb, Vijayan K. Asari

Vijayan K. Asari

In this paper we present a low level image descriptor called Histogram of Oriented Phase based on phase congruency concept and the Principal Component Analysis (PCA). Since the phase of the signal conveys more information regarding signal structure than the magnitude, the proposed descriptor can precisely identify and localize image features over the gradient based techniques, especially in the regions affected by illumination changes. The proposed features can be formed by extracting the phase congruency information for each pixel in the image with respect to its neighborhood. Histograms of the phase congruency values of the local regions in the image …

Intensity And Resolution Enhancement Of Local Regions For Object Detection And Tracking In Wide Area Surveillance, Evan Krieger, Vijayan K. Asari, Saibabu Arigela, Theus H. Aspiras

Vijayan K. Asari

Object tracking in wide area motion imagery is a complex problem that consists of object detection and target tracking over time. This challenge can be solved by human analysts who naturally have the ability to keep track of an object in a scene. A computer vision solution for object tracking has the potential to be a much faster and efficient solution. However, a computer vision solution faces certain challenges that do not affect a human analyst. To overcome these challenges, a tracking process is proposed that is inspired by the known advantages of a human analyst. First, the focus of …

Gaussian Nonlinear Line Attractor For Learning Multidimensional Data, Theus H. Aspiras, Vijayan K. Asari, Wesam Sakla

Vijayan K. Asari

The human brain’s ability to extract information from multidimensional data modeled by the Nonlinear Line Attractor (NLA), where nodes are connected by polynomial weight sets. Neuron connections in this architecture assumes complete connectivity with all other neurons, thus creating a huge web of connections. We envision that each neuron should be connected to a group of surrounding neurons with weighted connection strengths that reduces with proximity to the neuron. To develop the weighted NLA architecture, we use a Gaussian weighting strategy to model the proximity, which will also reduce the computation times significantly. Once all data has been trained in …

Automatic Building Change Detection In Wide Area Surveillance, Paheding Sidike, Almabrok Essa, Fatema Albalooshi, Vijayan K. Asari, Varun Santhaseelan

Vijayan K. Asari

We present an automated mechanism that can detect and characterize the building changes by analyzing airborne or satellite imagery. The proposed framework can be categorized into three stages: building detection, boundary extraction and change identification. To detect the buildings, we utilize local phase and local amplitude from monogenic signal to extract building features for addressing issues of varying illumination. Then a support vector machine with Radial basis kernel is used for classification. In the boundary extraction stage, a level-set function with self-organizing map based segmentation method is used to find the building boundary and compute physical area of the building …

Brain Machine Interface Using Emotiv Epoc To Control Robai Cyton Robotic Arm, Daniel P. Prince, Mark J. Edmonds, Andrew J. Sutter, Matthew Thomas Cusumano, Wenjie Lu, Vijayan K. Asari

Vijayan K. Asari

The initial framework for an electroencephalography (EEG) thought recognition software suite is developed, built, and tested. This suite is designed to recognize human thoughts and pair them to actions for controlling a robotic arm. Raw EEG brain activity data is collected using an Emotiv EPOC headset. The EEG data is processed through linear discriminant analysis (LDA), where an intended action is identified. The EEG classification suite is being developed to increase the number of distinct actions that can be identified compared to the Emotiv recognition software. The EEG classifier was able to correctly distinguish between two separate physical movements. Future …

A Modular Approach For Key-Frame Selection In Wide Area Surveillance Video Analysis, Almabrok Essa, Paheding Sidike, Vijayan K. Asari

Vijayan K. Asari

This paper presents an efficient preprocessing algorithm for big data analysis. Our proposed key-frame selection method utilizes the statistical differences among subsequent frames to automatically select only the frames that contain the desired contextual information and discard the rest of the insignificant frames.

We anticipate that such key frame selection technique will have significant impact on wide area surveillance applications such as automatic object detection and recognition in aerial imagery. Three real-world datasets are used for evaluation and testing and the observed results are encouraging.

A Robust Fringe-Adjusted Joint Transform Correlator For Efficient Object Detection, Paheding Sidike, Vijayan K. Asari, Mohammad S. Alam

Vijayan K. Asari

The fringe-adjusted joint transform correlation (FJTC) technique has been widely used for real-time optical pattern recognition applications. However, the classical FJTC technique suffers from target distortions due to noise, scale, rotation and illumination variations of the targets in input scenes. Several improvements of the FJTC have been proposed in the literature to accommodate these problems. Some popular techniques such as synthetic discriminant function (SDF) based FJTC was designed to alleviate the problems of scale and rotation variations of the target, whereas wavelet based FJTC has been found to yield better performance for noisy targets in the input scenes. While these …