Computer Engineering Commons^™

Signal Processing And Data Analysis For Real-Time Intermodal Freight Classification Through A Multimodal Sensor System., Enrique J. Sanchez Headley

Graduate Theses and Dissertations

Identifying freight patterns in transit is a common need among commercial and municipal entities. For example, the allocation of resources among Departments of Transportation is often predicated on an understanding of freight patterns along major highways. There exist multiple sensor systems to detect and count vehicles at areas of interest. Many of these sensors are limited in their ability to detect more specific features of vehicles in traffic or are unable to perform well in adverse weather conditions. Despite this limitation, to date there is little comparative analysis among Laser Imaging and Detection and Ranging (LIDAR) sensors for freight detection …

Towards Semantic Integration Of Machine Vision Systems To Aid Manufacturing Event Understanding, Kaishu Xia, Clint Saidy, Max Kirkpatrick, Noble Anumbe, Amit Sheth, Ramy Harik

Publications

A manufacturing paradigm shift from conventional control pyramids to decentralized, service-oriented, and cyber-physical systems (CPSs) is taking place in today’s 4th industrial revolution. Generally accepted roles and implementation recipes of cyber systems are expected to be standardized in the future of manufacturing industry. The authors intend to develop a novel CPS-enabled control architecture that accommodates: (1) intelligent information systems involving domain knowledge, empirical model, and simulation; (2) fast and secured industrial communication networks; (3) cognitive automation by rapid signal analytics and machine learning (ML) based feature extraction; (4) interoperability between machine and human. Semantic integration of process indicators is fundamental …

A Quantitative Validation Of Multi-Modal Image Fusion And Segmentation For Object Detection And Tracking, Nicholas Lahaye, Michael J. Garay, Brian D. Bue, Hesham El-Askary, Erik Linstead

Mathematics, Physics, and Computer Science Faculty Articles and Research

In previous works, we have shown the efficacy of using Deep Belief Networks, paired with clustering, to identify distinct classes of objects within remotely sensed data via cluster analysis and qualitative analysis of the output data in comparison with reference data. In this paper, we quantitatively validate the methodology against datasets currently being generated and used within the remote sensing community, as well as show the capabilities and benefits of the data fusion methodologies used. The experiments run take the output of our unsupervised fusion and segmentation methodology and map them to various labeled datasets at different levels of global …

Machine Learning-Based Recognition On Crowdsourced Food Images, Aditya Kulkarni

Honors Scholar Theses

With nearly a third of the world’s population suffering from food-induced chronic diseases such as obesity, the role of food in community health is required now more than ever. While current research underscores food proximity and density, there is a dearth in regard to its nutrition and quality. However, recent research in geospatial data collection and analysis as well as intelligent deep learning will help us study this further.

Employing the efficiency and interconnection of computer vision and geospatial technology, we want to study whether healthy food in the community is attainable. Specifically, with the help of deep learning in …

Analysis Of Recent Trends In Continuous Sign Language Recognition Using Nlp, Vijayshri Nitin Khedkar, Sonali Kothari Dr, Aarohi Prasad, Arunima Mishra, Varun Saha, Vinay Kumar

Library Philosophy and Practice (e-journal)

Oralism is an ideology and practice that advocates communication that is based solely on speech. This practice is encouraged from a pretty early age in our country. As a consequence, the hard of hearing are constantly forced to negotiate with schools, colleges, organisations, workspaces, and families that don’t acknowledge the need and preference for sign language over oral languages. This results in inconsideration of an entire community for admissions, jobs and general social position. We aim to close that communication gap a little and take a step towards fighting the stigma associated with Sign Language. The aim is to provide …

Accelerating Point Set Registration For Automated Aerial Refueling, Ryan M. Raettig

Theses and Dissertations

The goal of AAR is to control the tanker boom to safely refuel a receiving aircraft with no input or aid from the boom operator. To achieve this, the pose of the receiver relative to the tanker must be known. Point set registration is a fundamental issue used to estimate the relative pose of an object in an environment. However, it's likely a computational bottleneck of a vision processing pipeline. In addition, the matching of each sensed point with a closest truth point, nearest neighbor matching, is the most costly portion of the point set registration process. For this reason, …

Evaluating The Performance Of Transformer Architecture Over Attention Architecture On Image Captioning, Deepti Balasubramaniam

Dissertations

Over the last few decades computer vision and Natural Language processing has shown tremendous improvement in different tasks such as image captioning, video captioning, machine translation etc using deep learning models. However, there were not much researches related to image captioning based on transformers and how it outperforms other models that were implemented for image captioning. In this study will be designing a simple encoder-decoder model, attention model and transformer model for image captioning using Flickr8K dataset where will be discussing about the hyperparameters of the model, type of pre-trained model used and how long the model has been trained. …

‘Digits’ App - Smartphone Augmented Reality For Hand Telerehabilitation, Hongdao Dong, Edward Ho, Herbert Shin, Tania Banerjee, Geoffrey Masschelein, Jacob Davidson, Sandrine De Ribaupierre, Roy Eagleson, Caitlin Symonette

Electrical and Computer Engineering Publications

Hand telerehabilitation currently has limitations for accurate and remote assessment of range of motion (ROM) in small finger joints. ‘DIGITS’ application utilises the front smartphone camera to measure finger ROM in a reliable and rapid assessment protocol. Our initial beta-phase testing examined the consistency of our software measurements to in-person goniometry. 6 to 9 degrees of difference existed between the smartphone application recorded data versus the in-person measurements. This range is within acceptable 7 to 9 degree tolerance for interrater goniometry measurements. The effect of environmental factors such as hand distance, lightings and hand orientation was evaluated. The intraclass correlation …

Human-Robot Collaboration Enabled By Real-Time Vision Tracking, Travis Deegan

Electronic Theses and Dissertations

The number of robotic systems in the world is growing rapidly. However, most industrial robots are isolated in caged environments for the safety of users. There is an urgent need for human-in-the-loop collaborative robotic systems since robots are very good at performing precise and repetitive tasks but lack the cognitive ability and soft skills of humans. To fill this need, a key challenge is how to enable a robot to interpret its human co-worker’s motion and intention. This research addresses this challenge by developing a collaborative human-robot interface via innovations in computer vision, robotics, and system integration techniques. Specifically, this …

Ship Deck Segmentation In Engineering Document Using Generative Adversarial Networks, Mohammad Shahab Uddin, Raphael Pamie-George, Daron Wilkins, Andres Sousa Poza, Mustafa Canan, Samuel Kovacic, Jiang Li

Engineering Management & Systems Engineering Faculty Publications

Generative adversarial networks (GANs) have become very popular in recent years. GANs have proved to be successful in different computer vision tasks including image-translation, image super-resolution etc. In this paper, we have used GAN models for ship deck segmentation. We have used 2D scanned raster images of ship decks provided by US Navy Military Sealift Command (MSC) to extract necessary information including ship walls, objects etc. Our segmentation results will be helpful to get vector and 3D image of a ship that can be later used for maintenance of the ship. We applied the trained models to engineering documents provided …

Bibliometric Review On Image Based Plant Phenotyping, Shrikrishna Ulhas Kolhar, Jayant Jagtap

Library Philosophy and Practice (e-journal)

Plant phenotyping is a quantitative description of structural, physiological and temporal traits of plants resulting from interaction of plant genotypes with the environment. A rapid development is in progress in the field of image-based plant phenotyping. Plant phenotyping has wide range of applications in plant breeding research, plant growth prediction, biotic and abiotic stress analysis, crop management and early disease detection. The main motive is to provide detailed bibliometric review in order to know the available literature and current research trends in the area of plant phenotyping using plant images. The bibliometric analysis is primarily based on Scopus, web of …

A Deep Transfer Learning Based Model For Automatic Detection Of Covid-19from Chest X-Rays, Prateek Chhikara, Prakhar Gupta, Prabhjot Singh, Tarunpreet Bhatia

Turkish Journal of Electrical Engineering and Computer Sciences

Deep learning in medical imaging has revolutionized the way we interpret medical data, as high computational devices' capabilities are far more than their creators. With the pandemic causing havoc for the second straight year, the findings in our paper will allow researchers worldwide to use and create state-of-the-art models to detect affected persons before it reaches the R number. The paper proposes an automated diagnostic tool using the deep learning models on chest x-rays as an input to reach a point where we surpass this pandemic (COVID-19 disease). A deep transfer learning-based model for automatic detection of COVID-19 from chest …

Survey On Deep Neural Networks In Speech And Vision Systems, M. Alam, Manar D. Samad, Lasitha Vidyaratne, ‪Alexander Glandon, Khan M. Iftekharuddin

Computer Science Faculty Research

This survey presents a review of state-of-the-art deep neural network architectures, algorithms, and systems in speech and vision applications. Recent advances in deep artificial neural network algorithms and architectures have spurred rapid innovation and development of intelligent speech and vision systems. With availability of vast amounts of sensor data and cloud computing for processing and training of deep neural networks, and with increased sophistication in mobile and embedded technology, the next-generation intelligent systems are poised to revolutionize personal and commercial computing. This survey begins by providing background and evolution of some of the most successful deep learning models for intelligent …

Navigating In Numerous Video Data: User Interface Design For An On-Camera Video Analytics Engine, Sabriya Maryam Alam

The Journal of Purdue Undergraduate Research

Video analytics powered by artificial intelligence shows high promise in making our society smarter. Harnessing large amounts of video data, however, requires the development of processing systems demonstrating high performance and high efficiency. To this end, this work has contributed to a video analytics system powered by artificial intelligence for object detection and recognition. Rather than streaming all the video frames to the cloud, the system analyzes images on-camera and only returns those of interest to the cloud. This edge analytics research-grade software is available, but it lacks a simple web interface for general use by scientists, engineers, and other …

Study On Irregular Projection Surface Depth Perception Geometric Correction, Baoxing Bai, Yang Fan, Han Cheng, Zhang Chao

Journal of System Simulation

Abstract: In order to improve the universality of the projector projection scene, a method of depth perception geometric correction on the irregular projection surface was proposed. With the theoretical analysis of the coupling, the weaken coupling solution was proposed. The depth of the irregular projection surface could be acquired by the encoded color structured light. After that, the feedback was used to solve the maximum visual projection area, the dynamic slicing algorithm for the irregular surfaces depth information was used to pull away the stratification planes, so a corresponding homography matrix for each stratification plane was calculated and the homography …

Monocular Depth Image Mark-Less Pose Estimation Based On Feature Regression, Chen Ying, Shen Li

Journal of System Simulation

Abstract: Monocular camera mark-less pose estimation system suffers low accuracy, robustness and efficiency due to variety of action, self-occlusion of human body. A method of feature exaction from point clouds was proposed, in which a single-to-multiple (S2M) feature regressor and a joint position regressor were designed to quickly and accurately predict the 3D positions of body joints from a single depth image without any temporal information. Experiment result shows that the estimation accuracy is superior to that of state-of-the-arts and multi-camera based methods.

Automated Recognition Of Facial Affect Using Deep Neural Networks, Behzad Hasani

Electronic Theses and Dissertations

Automated Facial Expression Recognition (FER) has been a topic of study in the field of computer vision and machine learning for decades. In spite of efforts made to improve the accuracy of FER systems, existing methods still are not generalizable and accurate enough for use in real-world applications. Many of the traditional methods use hand-crafted (a.k.a. engineered) features for representation of facial images. However, these methods often require rigorous hyper-parameter tuning to achieve favorable results.

Recently, Deep Neural Networks (DNNs) have shown to outperform traditional methods in visual object recognition. DNNs require huge data as well as powerful computing units …

Past To Present (P2p): Road Thermal Image Colorization, Yuseong Park

Electronic Theses and Dissertations

Thermal image colorization into realistic RGB image is a challenging task. Thermal cameras are easily to detect objects in particular situation (e.g. darkness and fog) that the human eyes cannot detect. However, it is difficult to interpret the thermal image with human eyes. Enhancing thermal image colorization is an important task to improve these areas. The results of the existing colorization method still have color ambiguities, distortion, and blurriness problems. This paper focused on thermal image colorization using pix2pix network architecture based on Generative Adversarial Net (GAN). Pix2pix is a model that transforms thermal image into RGB image, but our …

Design Of A Novel Wearable Ultrasound Vest For Autonomous Monitoring Of The Heart Using Machine Learning, Garrett G. Goodman

Browse all Theses and Dissertations

As the population of older individuals increases worldwide, the number of people with cardiovascular issues and diseases is also increasing. The rate at which individuals in the United States of America and worldwide that succumb to Cardiovascular Disease (CVD) is rising as well. Approximately 2,303 Americans die to some form of CVD per day according to the American Heart Association. Furthermore, the Center for Disease Control and Prevention states that 647,000 Americans die yearly due to some form of CVD, which equates to one person every 37 seconds. Finally, the World Health Organization reports that the number one cause of …

Vegetation Detection Using Deep Learning And Conventional Methods, Bulent Ayhan, Chiman Kwan, Bence Budavari, Liyun Kwan, Yan Lu, Daniel Perez, Jiang Li, Dimitrios Skarlatos, Marinos Vlachos

Electrical & Computer Engineering Faculty Publications

Land cover classification with the focus on chlorophyll-rich vegetation detection plays an important role in urban growth monitoring and planning, autonomous navigation, drone mapping, biodiversity conservation, etc. Conventional approaches usually apply the normalized difference vegetation index (NDVI) for vegetation detection. In this paper, we investigate the performance of deep learning and conventional methods for vegetation detection. Two deep learning methods, DeepLabV3+ and our customized convolutional neural network (CNN) were evaluated with respect to their detection performance when training and testing datasets originated from different geographical sites with different image resolutions. A novel object-based vegetation detection approach, which utilizes NDVI, computer …

Deep Temporal Motion Descriptor (Dtmd) For Human Action Recognition, Nudrat Nida, Muhammad Haroon Yousaf, Aun Irtaza, Sergio A. Velastin

Turkish Journal of Electrical Engineering and Computer Sciences

Spatiotemporal features have significant importance in human action recognition, as they provide the actor's shape and motion characteristics specific to each action class. This paper presents a new deep spatiotemporal human action representation, the deep temporal motion descriptor (DTMD), which shares the attributes of holistic and deep learned features. To generate the DTMD descriptor, the actor?s silhouettes are gathered into single motion templates by applying motion history images. These motion templates capture the spatiotemporal movements of the actor and compactly represent the human actions using a single 2D template. Then deep convolutional neural networks are used to compute discriminative deep …

Amodal Instance Segmentation And Multi-Object Tracking With Deep Pixel Embedding, Yanfeng Liu

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

This thesis extends upon the representational output of semantic instance segmentation by explicitly including both visible and occluded parts. A fully convolutional network is trained to produce consistent pixel-level embedding across two layers such that, when clustered, the results convey the full spatial extent and depth ordering of each instance. Results demonstrate that the network can accurately estimate complete masks in the presence of occlusion and outperform leading top-down bounding-box approaches.

The model is further extended to produce consistent pixel-level embeddings across two consecutive image frames from a video to simultaneously perform amodal instance segmentation and multi-object tracking. No post-processing …

The Applications Of Grid Cells In Computer Vision, Keaton Kraiger

Maseeh Summer Undergraduate Research Experience

In this study we present a novel method for position and scale invariant object representation based on a biologically-inspired framework. Grid cells are neurons in the entorhinal cortex whose multiple firing locations form a periodic triangular array, tiling the surface of an animal’s environment. We propose a model for simple object representation that maintains position and scale invariance, in which grid maps capture the fundamental structure and features of an object. The model provides a mechanism for identifying feature locations in a Cartesian plane and vectors between object features encoded by grid cells. It is shown that key object features …

Infrared And Electro-Optical Stereo Vision For Automated Aerial Refueling, William E. Dallmann

Theses and Dissertations

Currently, Unmanned Aerial Vehicles are unsafe to refuel in-flight due to the communication latency between the UAVs ground operator and the UAV. Providing UAVs with an in-flight refueling capability would improve their functionality by extending their flight duration and increasing their flight payload. Our solution to this problem is Automated Aerial Refueling (AAR) using stereo vision from stereo electro-optical and infrared cameras on a refueling tanker. To simulate a refueling scenario, we use ground vehicles to simulate a pseudo tanker and pseudo receiver UAV. Imagery of the receiver is collected by the cameras on the tanker and processed by a …

American Sign Language Recognition Using Machine Learning And Computer Vision, Kshitij Bantupalli, Ying Xie

Master of Science in Computer Science Theses

Speech impairment is a disability which affects an individual’s ability to communicate using speech and hearing. People who are affected by this use other media of communication such as sign language. Although sign language is ubiquitous in recent times, there remains a challenge for non-sign language speakers to communicate with sign language speakers or signers. With recent advances in deep learning and computer vision there has been promising progress in the fields of motion and gesture recognition using deep learning and computer vision-based techniques. The focus of this work is to create a vision-based application which offers sign language translation …

Depth Enhancement And Surface Reconstruction With Rgb/D Sequence, Xinxin Zuo

Theses and Dissertations--Computer Science

Surface reconstruction and 3D modeling is a challenging task, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. It is fundamental to many applications such as robot navigation, animation and scene understanding, industrial control and medical diagnosis. In this dissertation, I take advantage of the consumer depth sensors for surface reconstruction. Considering its limited performance on capturing detailed surface geometry, a depth enhancement approach is proposed in the first place to recovery small and rich geometric details with captured depth and color sequence. In addition to enhancing its spatial resolution, I present a …

Recognition Of Incomplete Objects Based On Synthesis Of Views Using A Geometric Based Local-Global Graphs, Michael Christopher Robbeloth

Browse all Theses and Dissertations

The recognition of single objects is an old research field with many techniques and robust results. The probabilistic recognition of incomplete objects, however, remains an active field with challenging issues associated to shadows, illumination and other visual characteristics. With object incompleteness, we mean missing parts of a known object and not low-resolution images of that object. The employment of various single machine-learning methodologies for accurate classification of the incomplete objects did not provide a robust answer to the challenging problem. In this dissertation, we present a suite of high-level, model-based computer vision techniques encompassing both geometric and machine learning approaches …

Multi-Pig Part Detection And Association With A Fully-Convolutional Network, Eric T. Psota, Mateusz Mittek, Lance C. Pérez, Ty Schmidt, Benny Mote

Department of Electrical and Computer Engineering: Faculty Publications

Computer vision systems have the potential to provide automated, non-invasive monitoring of livestock animals, however, the lack of public datasets with well-defined targets and evaluation metrics presents a significant challenge for researchers. Consequently, existing solutions often focus on achieving task-specific objectives using relatively small, private datasets. This work introduces a new dataset and method for instance-level detection of multiple pigs in group-housed environments. The method uses a single fully-convolutional neural network to detect the location and orientation of each animal, where both body part locations and pairwise associations are represented in the image space. Accompanying this method is a new …

Elimination Of Useless Images From Raw Camera-Trap Data, Ulaş Tekeli̇, Yalin Baştanlar

Turkish Journal of Electrical Engineering and Computer Sciences

Camera-traps are motion triggered cameras that are used to observe animals in nature. The number of images collected from camera-traps has increased significantly with the widening use of camera-traps thanks to advances in digital technology. A great workload is required for wild-life researchers to group and label these images. We propose a system to decrease the amount of time spent by the researchers by eliminating useless images from raw camera-trap data. These images are too bright, too dark, blurred, or they contain no animals. To eliminate bright, dark, and blurred images we employ techniques based on image histograms and fast …

Enhancing 3d Visual Odometry With Single-Camera Stereo Omnidirectional Systems, Carlos A. Jaramillo

Dissertations, Theses, and Capstone Projects

We explore low-cost solutions for efficiently improving the 3D pose estimation problem of a single camera moving in an unfamiliar environment. The visual odometry (VO) task -- as it is called when using computer vision to estimate egomotion -- is of particular interest to mobile robots as well as humans with visual impairments. The payload capacity of small robots like micro-aerial vehicles (drones) requires the use of portable perception equipment, which is constrained by size, weight, energy consumption, and processing power. Using a single camera as the passive sensor for the VO task satisfies these requirements, and it motivates the …