Computer Sciences Commons^™

Reciprocal Transformations For Unsupervised Video Object Segmentation, Sucheng Ren, Wenxi Liu, Yongtuo Liu, Haoxin Chen, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Unsupervised video object segmentation (UVOS) aims at segmenting the primary objects in videos without any human intervention. Due to the lack of prior knowledge about the primary objects, identifying them from videos is the major challenge of UVOS. Previous methods often regard the moving objects as primary ones and rely on optical flow to capture the motion cues in videos, but the flow information alone is insufficient to distinguish the primary objects from the background objects that move together. This is because, when the noisy motion features are combined with the appearance features, the localization of the primary objects is …

Adaptive Aggregation Networks For Class-Incremental Learning, Yaoyao Liu, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

Class-Incremental Learning (CIL) aims to learn a classification model with the number of classes increasing phase-by-phase. An inherent problem in CIL is the stability-plasticity dilemma between the learning of old and new classes, i.e., high-plasticity models easily forget old classes, but high-stability models are weak to learn new classes. We alleviate this issue by proposing a novel network architecture called Adaptive Aggregation Networks (AANets) in which we explicitly build two types of residual blocks at each residual level (taking ResNet as the baseline architecture): a stable block and a plastic block. We aggregate the output feature maps from these two …

Rm-Net: Rasterizing Markov Signals To Images For Deep Learning, Kajal Gupta

Theses

Statistical machine learning approaches are quite famous for processing Markov signal data. They can model unobserved states and learn certain characteristics particular to a signal with good accuracy. However, with the advent of Deep learning the novice ways of solving a problem has shifted towards this more sophisticated algorithm, which is much better, powerful and more accurate. Specifically, Convolutional Neural Nets (CNN) have shown many promising results on images and videos. Here we illustrate how CNN can be applied to a 1D numeric signal using signal rasterization technique. We start by rasterizing a 1D numeric Markov signal into an image …

Towards Open World Object Detection, K. J. Joseph, Salman Khan, Fahad Shahbaz Khan, Vineeth N. Balasubramanian

Computer Vision Faculty Publications

Humans have a natural instinct to identify unknown object instances in their environments. The intrinsic curiosity about these unknown instances aids in learning about them, when the corresponding knowledge is eventually available. This motivates us to propose a novel computer vision problem called: 'Open World Object Detection', where a model is tasked to: 1) identify objects that have not been introduced to it as 'unknown', without explicit supervision to do so, and 2) incrementally learn these identified unknown categories without forgetting previously learned classes, when the corresponding labels are progressively received. We formulate the problem, introduce a strong evaluation protocol …

Using Deep Learning To Analyze Materials In Medical Images, Carson Molder

Computer Science and Computer Engineering Undergraduate Honors Theses

Modern deep learning architectures have become increasingly popular in medicine, especially for analyzing medical images. In some medical applications, deep learning image analysis models have been more accurate at predicting medical conditions than experts. Deep learning has also been effective for material analysis on photographs. We aim to leverage deep learning to perform material analysis on medical images. Because material datasets for medicine are scarce, we first introduce a texture dataset generation algorithm that automatically samples desired textures from annotated or unannotated medical images. Second, we use a novel Siamese neural network called D-CNN to predict patch similarity and build …

Regularized Deep Network Learning For Multi-Label Visual Recognition, Hao Guo

Theses and Dissertations

This dissertation is focused on the task of multi-label visual recognition, a fundamental task of computer vision. It aims to tell the presence of multiple visual classes from the input image, where the visual classes, such as objects, scenes, attributes, etc., are usually defined as image labels. Due to the prosperous deep networks, this task has been widely studied and significantly improved in recent years. However, it remains a challenging task due to appearance complexity of multiple visual contents co-occurring in one image. This research explores to regularize the deep network learning for multi-label visual recognition.

First, an attention concentration …

Automatic Detection Of Vehicles In Satellite Images For Economic Monitoring, Cole Hill

USF Tampa Graduate Theses and Dissertations

With the growing supply of satellites capturing images of the planet, governments andinvestors are looking for ways in which these new images may be used to determine which businesses are struggling and thriving. Recent works have shown that parking lot fill rates can provide valuable information about businesses’ earnings, however, the task of manually annotating the number of vehicles in a parking lot is expensive and time-consuming. Systems which can automate this process are therefore valuable as they are faster and cheaper than human labor. In this thesis, the problem of detection of small objects in large low-resolution images is …

Stereo Camera Calibrations With Optical Flow, Joshua D. Larson

Theses and Dissertations

Remotely Piloted Aircraft (RPA) are currently unable to refuel mid-air due to the large communication delays between their operators and the aircraft. AAR seeks to address this problem by reducing the communication delay to a fast line-of-sight signal between the tanker and the RPA. Current proposals for AAR utilize stereo cameras to estimate where the receiving aircraft is relative to the tanker, but require accurate calibrations for accurate location estimates of the receiver. This paper improves the accuracy of this calibration by improving three components of it: increasing the quantity of intrinsic calibration data with CNN preprocessing, improving the quality …

Accurate Covariance Estimation For Pose Data From Iterative Closest Point Algorithm, Rick H. Yuan

Theses and Dissertations

One of the fundamental problems of robotics and navigation is the estimation of relative pose of an external object with respect to the observer. A common method for computing the relative pose is the Iterative Closest Point (ICP) algorithm, where a reference point cloud of a known object is registered against a sensed point cloud to determine relative pose. To use this computed pose information in down-stream processing algorithms, it is necessary to estimate the uncertainty of the ICP output, typically represented as a covariance matrix. In this thesis a novel method for estimating uncertainty from sensed data is introduced. …

Multi-Modal Classification Using Images And Text, Stuart J. Miller, Justin Howard, Paul Adams, Mel Schwan, Robert Slater

SMU Data Science Review

This paper proposes a method for the integration of natural language understanding in image classification to improve classification accuracy by making use of associated metadata. Traditionally, only image features have been used in the classification process; however, metadata accompanies images from many sources. This study implemented a multi-modal image classification model that combines convolutional methods with natural language understanding of descriptions, titles, and tags to improve image classification. The novelty of this approach was to learn from additional external features associated with the images using natural language understanding with transfer learning. It was found that the combination of ResNet-50 image …

Evaluating The Performance Of Transformer Architecture Over Attention Architecture On Image Captioning, Deepti Balasubramaniam

Dissertations

Over the last few decades computer vision and Natural Language processing has shown tremendous improvement in different tasks such as image captioning, video captioning, machine translation etc using deep learning models. However, there were not much researches related to image captioning based on transformers and how it outperforms other models that were implemented for image captioning. In this study will be designing a simple encoder-decoder model, attention model and transformer model for image captioning using Flickr8K dataset where will be discussing about the hyperparameters of the model, type of pre-trained model used and how long the model has been trained. …

View Synthesis Of Dynamic Scenes Based On Deep 3d Mask Volume, Kai-En Lin, Guowei Yang, Lei Xiao, Feng Liu, Ravi Ramamoorthi

Computer Science Faculty Publications and Presentations

Image view synthesis has seen great success in reconstructing photorealistic visuals, thanks to deep learning and various novel representations. The next key step in immersive virtual experiences is view synthesis of dynamic scenes. However, several challenges exist due to the lack of high-quality training datasets, and the additional time dimension for videos of dynamic scenes. To address this issue, we introduce a multi-view video dataset, captured with a custom 10-camera rig in 120FPS. The dataset contains 96 high-quality scenes showing various visual effects and human interactions in outdoor scenes. We develop a new algorithm, Deep 3D Mask Volume, which enables …

Learning Graphs For Object Tracking And Counting, Shengkun Li

Legacy Theses & Dissertations (2009 - 2024)

As important problems in computer vision, object tracking and counting attract increasing amounts of attention in recent years due to its wide range of applications, such as video surveillance, human- computer interaction, smart city. Despite much progress has been made in object tracking and counting with the arriving of deep neural networks (DNN), there still remains much room for improvement to satisfy the real-world applications.

Ship Deck Segmentation In Engineering Document Using Generative Adversarial Networks, Mohammad Shahab Uddin, Raphael Pamie-George, Daron Wilkins, Andres Sousa Poza, Mustafa Canan, Samuel Kovacic, Jiang Li

Engineering Management & Systems Engineering Faculty Publications

Generative adversarial networks (GANs) have become very popular in recent years. GANs have proved to be successful in different computer vision tasks including image-translation, image super-resolution etc. In this paper, we have used GAN models for ship deck segmentation. We have used 2D scanned raster images of ship decks provided by US Navy Military Sealift Command (MSC) to extract necessary information including ship walls, objects etc. Our segmentation results will be helpful to get vector and 3D image of a ship that can be later used for maintenance of the ship. We applied the trained models to engineering documents provided …

Inference Of Surface Velocities From Oblique Time Lapse Photos And Terrestrial Based Lidar At The Helheim Glacier, Franklyn T. Dunbar Ii

Graduate Student Theses, Dissertations, & Professional Papers

Using time dependent observations derived from terrestrial LiDAR and oblique
time-lapse imagery, we demonstrate that a Bayesian approach to glacial motion es-
timation provides a concise way to incorporate multiple data products into a single
motion estimation procedure effectively producing surface velocity estimates with
an associated uncertainty. This approach brings both improved computational effi-
ciency, and greater scalability across observational time-frames when compared to
existing methods. To gauge efficacy, we apply these methods to a set of observa-
tions from the Helheim Glacier, a critical actor in contemporary mass loss trends
observed in the Greenland Ice Sheet. We find that …

Multi-Branch Gabor Wavelet Layers For Pedestrian Attribute Recognition, Imran N. Junejo

All Works

CCBYNCND Surveillance cameras are everywhere, keeping an eye on pedestrians as they navigate through a scene. With this context, our paper addresses the problem of pedestrian attribute recognition (PAR). This problem entails recognizing attributes such as age-group, clothing style, accessories, footwear style etc. This is a multi-label problem and challenging even for human observers. The problem has rightly attracted attention recently from the computer vision community. In this paper, we adopt trainable Gabor wavelets (TGW) layers and use it with a convolution neural network (CNN). Whereas other researchers are using fixed Gabor filters with the CNN, the proposed layers are …

A Deep Transfer Learning Based Model For Automatic Detection Of Covid-19from Chest X-Rays, Prateek Chhikara, Prakhar Gupta, Prabhjot Singh, Tarunpreet Bhatia

Turkish Journal of Electrical Engineering and Computer Sciences

Deep learning in medical imaging has revolutionized the way we interpret medical data, as high computational devices' capabilities are far more than their creators. With the pandemic causing havoc for the second straight year, the findings in our paper will allow researchers worldwide to use and create state-of-the-art models to detect affected persons before it reaches the R number. The paper proposes an automated diagnostic tool using the deep learning models on chest x-rays as an input to reach a point where we surpass this pandemic (COVID-19 disease). A deep transfer learning-based model for automatic detection of COVID-19 from chest …

Deep Unsupervised Anomaly Detection, Tangqing Li, Zheng Wang, Siying Liu, Wen-Yan Lin

Research Collection School Of Computing and Information Systems

This paper proposes a novel method to detect anomalies in large datasets under a fully unsupervised setting. The key idea behind our algorithm is to learn the representation underlying normal data. To this end, we leverage the latest clustering technique suitable for handling high dimensional data. This hypothesis provides a reliable starting point for normal data selection. We train an autoencoder from the normal data subset, and iterate between hypothesizing normal candidate subset based on clustering and representation learning. The reconstruction error from the learned autoencoder serves as a scoring function to assess the normality of the data. Experimental results …

Survey On Deep Neural Networks In Speech And Vision Systems, M. Alam, Manar D. Samad, Lasitha Vidyaratne, ‪Alexander Glandon, Khan M. Iftekharuddin

Computer Science Faculty Research

This survey presents a review of state-of-the-art deep neural network architectures, algorithms, and systems in speech and vision applications. Recent advances in deep artificial neural network algorithms and architectures have spurred rapid innovation and development of intelligent speech and vision systems. With availability of vast amounts of sensor data and cloud computing for processing and training of deep neural networks, and with increased sophistication in mobile and embedded technology, the next-generation intelligent systems are poised to revolutionize personal and commercial computing. This survey begins by providing background and evolution of some of the most successful deep learning models for intelligent …

Semi-Automatic Hand Pose Estimation Using A Single Depth Camera, Giffy Jerald Chris

Computer Science and Engineering Theses - Archive

This paper addresses the problem of 3D hand pose annotations using a single depth camera. Although hand pose estimation methods rely critically on accurate 3D training data, creating such reliable training data is challenging and labor intensive. We propose a semi-automatic method for efficiently and accurately labeling the 3D hand key-points in a hand depth video. The process starts by selecting a subset of frames that are representative of all the frames in the dataset and the annotator only provides an estimate of the 2D hand key-points in these selected frames. We use this information to infer the 3D location …

Language-Driven Region Pointer Advancement For Controllable Image Captioning, Annika Lindh, Robert J. Ross, John D. Kelleher

Conference papers

Controllable Image Captioning is a recent sub-field in the multi-modal task of Image Captioning wherein constraints are placed on which regions in an image should be described in the generated natural language caption. This puts a stronger focus on producing more detailed descriptions, and opens the door for more end-user control over results. A vital component of the Controllable Image Captioning architecture is the mechanism that decides the timing of attending to each region through the advancement of a region pointer. In this paper, we propose a novel method for predicting the timing of region pointer advancement by treating the …

Enhancing Developer Interactions With Programming Screencasts Through Accurate Code Extraction, Lingfeng Bao, Shengyi Pan, Zhenchang Xing, Xin Xia, David Lo, Xiaohu Yang

Research Collection School Of Computing and Information Systems

Programming screencasts have become a pervasive resource on the Internet, which is favoured by many developers for learning new programming skills. For developers, the source code in screencasts is valuable and important. However, the streaming nature of screencasts limits the choice that they have for interacting with the code. Many studies apply the Optical Character Recognition (OCR) technique to convert screen images into text, which can be easily searched and indexed. However, we observe that the noise in the screen images significantly affects the quality of OCRed code.In this paper, we develop a tool named psc2code, which has two components, …

Modular Neural Networks For Low-Power Image Classification On Embedded Devices, Abhinav Goel, Sara Aghajanzadeh, Caleb Tung, Shuo-Han Chen, George K. Thiruvathukal, Yung-Hisang Lu

Computer Science: Faculty Publications and Other Works

Embedded devices are generally small, battery-powered computers with limited hardware resources. It is difficult to run deep neural networks (DNNs) on these devices, because DNNs perform millions of operations and consume significant amounts of energy. Prior research has shown that a considerable number of a DNN’s memory accesses and computation are redundant when performing tasks like image classification. To reduce this redundancy and thereby reduce the energy consumption of DNNs, we introduce the Modular Neural Network Tree architecture. Instead of using one large DNN for the classifier, this architecture uses multiple smaller DNNs (called modules) to progressively classify images …

Fakepolisher: Making Deepfakes More Detection-Evasive By Shallow Reconstruction, Yihao Huang, Felix Juefei-Xu, Run Wang, Qing Guo, Lei Ma, Xiaofei Xie, Jianwen Li, Weikai Miao, Yang Liu, Geguang Pu

Research Collection School Of Computing and Information Systems

At this moment, GAN-based image generation methods are still imperfect, whose upsampling design has limitations in leaving some certain artifact patterns in the synthesized image. Such artifact patterns can be easily exploited (by recent methods) for difference detection of real and GAN-synthesized images. However, the existing detection methods put much emphasis on the artifact patterns, which can become futile if such artifact patterns were reduced.Towards reducing the artifacts in the synthesized images, in this paper, we devise a simple yet powerful approach termed FakePolisher that performs shallow reconstruction of fake images through a learned linear dictionary, intending to effectively and …

Study On Irregular Projection Surface Depth Perception Geometric Correction, Baoxing Bai, Yang Fan, Han Cheng, Zhang Chao

Journal of System Simulation

Abstract: In order to improve the universality of the projector projection scene, a method of depth perception geometric correction on the irregular projection surface was proposed. With the theoretical analysis of the coupling, the weaken coupling solution was proposed. The depth of the irregular projection surface could be acquired by the encoded color structured light. After that, the feedback was used to solve the maximum visual projection area, the dynamic slicing algorithm for the irregular surfaces depth information was used to pull away the stratification planes, so a corresponding homography matrix for each stratification plane was calculated and the homography …

Novel View Synthesis - A Neural Network Approach, Hoang Le

Dissertations and Theses

Novel view synthesis is an important research problem in computer vision and computational photography. It enables a wide range of applications including re-cinematography, video enhancement, virtual reality, etc. These algorithms leverage a pre-acquired set of images taken from a set of viewpoints to synthesize another image at a novel viewpoint as if it was captured by a real camera. To synthesize a high-quality novel view, these algorithms often assume a static scene, or the images were captured synchronously. However, the scenes in practice are often dynamic, and taking a dense set of images of these scenes at the same moment …

Computer Vision Methods For Sign Language And Cognitive Evaluation Through Physical Tasks, Alex J. Dillhoff

Computer Science and Engineering Dissertations - Archive

Analyzing human motion is vital for a multitude of tasks including human-computer interaction, sign language recognition, and the assessment of cognitive disorders. Providing automatic assessments for cognitive disorders increases the accessibility and affordability of life-changing tests and treatments. For sign language recognition, automated translation systems bridge the gap between native and non-native signers. Additionally, dictionary look-up systems are helpful for native signers learning a new language. Common to both of these tasks is the reliance of fine motor function in the hands. Hand Pose Estimation methods are used to drive applications that rely on hand shape. These tasks present unique …

Hand-Over-Face Segmentation, Sakher Ghanem

Computer Science and Engineering Dissertations - Archive

Accurate hand segmentation is vital in many applications in which the hands play a central role, such as sign language recognition, action recognition, and gesture recognition. A relatively unexplored obstacle to correct hand segmentation is when the hand overlaps the face. The shortage of a dataset for this research area has been one motivation for this work. However, this dissertation investigates and proposes improvements for the hand-over-face segmentation task. Toward an in-depth study of the hand segmentation problem, the work presented in this dissertation will yield several contributions. First, it introduces a survey on sign language recognition systems using mobile …

Using Color Thresholding And Contouring To Understand Coral Reef Biodiversity, Scott Vuong Tran

Master's Projects

This paper presents research outcomes of understanding coral reef biodiversity through the usage of various computer vision applications and techniques. It aims to help further analyze and understand the coral reef biodiversity through the usage of color thresholding and contouring onto images of the ARMS plates to extract groups of microorganisms based on color. The results are comparable to the manual markup tool developed to do the same tasks and shows that the manual process can be sped up using computer vision. The paper presents an automated way to extract groups of microorganisms based on color without the use of …

Leveraging Model Flexibility And Deep Structure: Non-Parametric And Deep Models For Computer Vision Processes With Applications To Deep Model Compression, Anthony D. Rhodes

Dissertations and Theses

My dissertation presents several new algorithms incorporating non-parametric and deep learning approaches for computer vision and related tasks, including object localization, object tracking and model compression. With respect to object localization, I introduce a method to perform active localization by modeling spatial and other relationships between objects in a coherent "visual situation" using a set of probability distributions. I further refine this approach with the Multipole Density Estimation with Importance Clustering (MIC-Situate) algorithm. Next, I formulate active, "situation" object search as a Bayesian optimization problem using Gaussian Processes. Using my Gaussian Process Context Situation Learning (GP-CL) algorithm, I demonstrate improved …