Physical Sciences and Mathematics Commons^™

Computer Vision-Based Traffic Sign Detection And Extraction: A Hybrid Approach Using Gis And Machine Learning, Zihao Wu

Electronic Theses and Dissertations

Traffic sign detection and positioning have drawn considerable attention because of the recent development of autonomous driving and intelligent transportation systems. In order to detect and pinpoint traffic signs accurately, this research proposes two methods. In the first method, geo-tagged Google Street View images and road networks were utilized to locate traffic signs. In the second method, both traffic signs categories and locations were identified and extracted from the location-based GoPro video. TensorFlow is the machine learning framework used to implement these two methods. To that end, 363 stop signs were detected and mapped accurately using the first method (Google …

Generative Adversarial Networks For Online Visual Object Tracking Systems, Ghsoun Zin

Theses and Dissertations (Comprehensive)

Object Tracking is one of the essential tasks in computer vision domain as it has numerous applications in various fields, such as human-computer interaction, video surveillance, augmented reality, and robotics. Object Tracking refers to the process of detecting and locating the target object in a series of frames in a video. The state-of-the-art for tracking-by-detection framework is typically made up of two steps to track the target object. The first step is drawing multiple samples near the target region of the previous frame. The second step is classifying each sample as either the target object or the background. Visual object …

Enhancing 3d Visual Odometry With Single-Camera Stereo Omnidirectional Systems, Carlos A. Jaramillo

Dissertations, Theses, and Capstone Projects

We explore low-cost solutions for efficiently improving the 3D pose estimation problem of a single camera moving in an unfamiliar environment. The visual odometry (VO) task -- as it is called when using computer vision to estimate egomotion -- is of particular interest to mobile robots as well as humans with visual impairments. The payload capacity of small robots like micro-aerial vehicles (drones) requires the use of portable perception equipment, which is constrained by size, weight, energy consumption, and processing power. Using a single camera as the passive sensor for the VO task satisfies these requirements, and it motivates the …

Object Localization, Segmentation, And Classification In 3d Images, Allan Zelener

Dissertations, Theses, and Capstone Projects

We address the problem of identifying objects of interest in 3D images as a set of related tasks involving localization of objects within a scene, segmentation of observed object instances from other scene elements, classifying detected objects into semantic categories, and estimating the 3D pose of detected objects within the scene. The increasing availability of 3D sensors motivates us to leverage large amounts of 3D data to train machine learning models to address these tasks in 3D images. Leveraging recent advances in deep learning has allowed us to develop models capable of addressing these tasks and optimizing these tasks jointly …

Leveraging Overhead Imagery For Localization, Mapping, And Understanding, Scott Workman

Theses and Dissertations--Computer Science

Ground-level and overhead images provide complementary viewpoints of the world. This thesis proposes methods which leverage dense overhead imagery, in addition to sparsely distributed ground-level imagery, to advance traditional computer vision problems, such as ground-level image localization and fine-grained urban mapping. Our work focuses on three primary research areas: learning a joint feature representation between ground-level and overhead imagery to enable direct comparison for the task of image geolocalization, incorporating unlabeled overhead images by inferring labels from nearby ground-level images to improve image-driven mapping, and fusing ground-level imagery with overhead imagery to enhance understanding. The ultimate contribution of this thesis …

Deep Probabilistic Models For Camera Geo-Calibration, Menghua Zhai

Theses and Dissertations--Computer Science

The ultimate goal of image understanding is to transfer visual images into numerical or symbolic descriptions of the scene that are helpful for decision making. Knowing when, where, and in which direction a picture was taken, the task of geo-calibration makes it possible to use imagery to understand the world and how it changes in time. Current models for geo-calibration are mostly deterministic, which in many cases fails to model the inherent uncertainties when the image content is ambiguous. Furthermore, without a proper modeling of the uncertainty, subsequent processing can yield overly confident predictions. To address these limitations, we propose …

Modeling And Mapping Location-Dependent Human Appearance, Zachary Bessinger

Theses and Dissertations--Computer Science

Human appearance is highly variable and depends on individual preferences, such as fashion, facial expression, and makeup. These preferences depend on many factors including a person's sense of style, what they are doing, and the weather. These factors, in turn, are dependent upon geographic location and time. In our work, we build computational models to learn the relationship between human appearance, geographic location, and time. The primary contributions are a framework for collecting and processing geotagged imagery of people, a large dataset collected by our framework, and several generative and discriminative models that use our dataset to learn the relationship …

Quantitative Behavior Tracking Of Xenopus Laevis Tadpoles For Neurobiology Research, Alexander Hansen Hamme

Senior Projects Fall 2018

Xenopus laevis tadpoles are a useful animal model for neurobiology research because they provide a means to study the development of the brain in a species that is both physiologically well-understood and logistically easy to maintain in the laboratory. For behavioral studies, however, their individual and social swimming patterns represent a largely untapped trove of data, due to the lack of a computational tool that can accurately track multiple tadpoles at once in video feeds. This paper presents a system that was developed to accomplish this task, which can reliably track up to six tadpoles in a controlled environment, thereby …

Formresnet: Formatted Residual Learning For Image Restoration, Jianbo Jiao, Wei-Chih Tu, Shengfeng He

Research Collection School Of Computing and Information Systems

In this paper, we propose a deep CNN to tackle the image restoration problem by learning the structured residual. Previous deep learning based methods directly learn the mapping from corrupted images to clean images, and may suffer from the gradient exploding/vanishing problems of deep neural networks. We propose to address the image restoration problem by learning the structured details and recovering the latent clean image together, from the shared information between the corrupted image and the latent image. In addition, instead of learning the pure difference (corruption), we propose to add a 'residual formatting layer' to format the residual to …

Deshadownet: A Multi-Context Embedding Deep Network For Shadow Removal, Liangqiong Qu, Jiandong Tian, Shengfeng He, Yandong Tang, Rynson W. H. Lau

Research Collection School Of Computing and Information Systems

Shadow removal is a challenging task as it requires the detection/annotation of shadows as well as semantic understanding of the scene. In this paper, we propose an automatic and end-to-end deep neural network (DeshadowNet) to tackle these problems in a unified manner. DeshadowNet is designed with a multi-context architecture, where the output shadow matte is predicted by embedding information from three different perspectives. The first global network extracts shadow features from a global view. Two levels of features are derived from the global network and transferred to two parallel networks. While one extracts the appearance of the input image, the …

Bayesian Optimization For Refining Object Proposals, With An Application To Pedestrian Detection, Anthony D. Rhodes

Student Research Symposium

We devise an algorithm using a Bayesian optimization framework in conjunction with contextual visual data for the efficient localization of objects in still images. Recent research has demonstrated substantial progress in object localization and related tasks for computer vision. However, many current state-of-the-art object localization procedures still suffer from inaccuracy and inefficiency, in addition to failing to successfully leverage contextual data. We address these issues with the current research.

Our method encompasses an active search procedure that uses contextual data to generate initial bounding-box proposals for a target object. We train a convolutional neural network to approximate an offset distance …

Indoor Scene Localization To Fight Sex Trafficking In Hotels, Abigail Stylianou

McKelvey School of Engineering Theses & Dissertations

Images are key to fighting sex trafficking. They are: (a) used to advertise for sex services,(b) shared among criminal networks, and (c) connect a person in an image to the place where the image was taken. This work explores the ability to link images to indoor places in order to support the investigation and prosecution of sex trafficking. We propose and develop a framework that includes a database of open-source information available on the Internet, a crowd-sourcing approach to gathering additional images, and explore a variety of matching approaches based both on hand-tuned features such as SIFT and learned features …

Vision-Based Motion For A Humanoid Robot, Khalid Abdullah Alkhulayfi

Dissertations and Theses

The overall objective of this thesis is to build an integrated, inexpensive, human-sized humanoid robot from scratch that looks and behaves like a human. More specifically, my goal is to build an android robot called Marie Curie robot that can act like a human actor in the Portland Cyber Theater in the play Quantum Debate with a known script of every robot behavior. In order to achieve this goal, the humanoid robot need to has degrees of freedom (DOF) similar to human DOFs. Each part of the Curie robot was built to achieve the goal of building a complete humanoid …

Exemplar-Driven Top-Down Saliency Detection Via Deep Association, Shengfeng He, Rynson W. H. Lau, Qingxiong Yang

Research Collection School Of Computing and Information Systems

Top-down saliency detection is a knowledge-driven search task. While some previous methods aim to learn this "knowledge" from category-specific data, others transfer existing annotations in a large dataset through appearance matching. In contrast, we propose in this paper a locateby-exemplar strategy. This approach is challenging, as we only use a few exemplars (up to 4) and the appearances among the query object and the exemplars can be very different. To address it, we design a two-stage deep model to learn the intra-class association between the exemplars and query objects. The first stage is for learning object-to-object association, and the second …

Collecting Image Cropping Dataset: A Hybrid System Of Machine And Human Intelligence, Uyen T. Mai, Feng Liu

Student Research Symposium

Image cropping is a common tool that exists in almost any image editor, yet automatic cropping is still a difficult problem in Computer Vision. Since images nowadays can be easily collected through the web, machine learning is a promising approach to solve this problem. However, an image cropping dataset is not yet available and gathering such a large-scale dataset is a non-trivial task. Although a crowdsourcing website such as Mechanical Turk seems to be a solution to this task, image cropping is a sophisticated task that is vulnerable to unreliable annotation; furthermore, collecting a large-scale high-quality dataset through crowdsourcing is …

Learning In Vision And Robotics, Daniel P. Barrett

Open Access Dissertations

I present my work on learning from video and robotic input. This is an important problem, with numerous potential applications. The use of machine learning makes it possible to obtain models which can handle noise and variation without explicitly programming them. It also raises the possibility of robots which can interact more seamlessly with humans rather than only exhibiting hard-coded behaviors. I will present my work in two areas: video action recognition, and robot navigation. First, I present a video action recognition method which represents actions in video by sequences of retinotopic appearance and motion detectors, learns such models automatically …

Grounding Robot Motion In Natural Language And Visual Perception, Scott Alan Bronikowski

Open Access Dissertations

The current state of the art in military and first responder ground robots involves heavy physical and cognitive burdens on the human operator while taking little to no advantage of the potential autonomy of robotic technology. The robots currently in use are rugged remote-controlled vehicles. Their interaction modalities, usually utilizing a game controller connected to a computer, require a dedicated operator who has limited capacity for other tasks.

I present research which aims to ease these burdens by incorporating multiple modes of robotic sensing into a system which allows humans to interact with robots through a natural-language interface. I conduct …

Oriented Object Proposals, Shengfeng He, Rynson W. H. Lau

Research Collection School Of Computing and Information Systems

In this paper, we propose a new approach to generate oriented object proposals (OOPs) to reduce the detection error caused by various orientations of the object. To this end, we propose to efficiently locate object regions according to pixelwise object probability, rather than measuring the objectness from a set of sampled windows. We formulate the proposal generation problem as a generative probabilistic model such that object proposals of different shapes (i.e., sizes and orientations) can be produced by locating the local maximum likelihoods. The new approach has three main advantages. First, it helps the object detector handle objects of different …

Automated Multi-Modal Search And Rescue Using Boosted Histogram Of Oriented Gradients, Matthew A. Lienemann

Master's Theses

Unmanned Aerial Vehicles (UAVs) provides a platform for many automated tasks and with an ever increasing advances in computing, these tasks can be more complex. The use of UAVs is expanded in this thesis with the goal of Search and Rescue (SAR), where a UAV can assist fast responders to search for a lost person and relay possible search areas back to SAR teams. To identify a person from an aerial perspective, low-level Histogram of Oriented Gradients (HOG) feature descriptors are used over a segmented region, provided from thermal data, to increase classification speed. This thesis also introduces a dataset …

Camera Viewpoint Invariant Vehicular Traffic Segmentation And Classification, Jeffrey B. Flora

Electrical & Computer Engineering Theses & Dissertations

The goal of this intelligent transportation systems work is to develop a computer vision method that is view angle independent for segmenting and classifying vehicular traffic on highway systems. In order to achieve this goal, this work implements an algorithm for vehicle segmentation, feature extraction, and classification using the existing Virginia Department of Transportation (VDOT) infrastructure on networked traffic cameras. The VDOT traffic video is analyzed for vehicle detection and segmentation using an adaptive Gaussian mixture model algorithm. Speed estimation is performed using a single camera calibration. Size and shape features from morphological properties and texture features from histogram of …

Using Gist Features To Constrain Search In Object Detection, Joanna Browne Solmon

Dissertations and Theses

This thesis investigates the application of GIST features [13] to the problem of object detection in images. Object detection refers to locating instances of a given object category in an image. It is contrasted with object recognition, which simply decides whether an image contains an object, regardless of the object's location in the image.

In much of computer vision literature, object detection uses a "sliding window" approach to finding objects in an image. This requires moving various sizes of windows across an image and running a trained classifier on the visual features of each window. This brute force method can …

3d Robotic Sensing Of People: Human Perception, Representation And Activity Recognition, Hao Zhang

Doctoral Dissertations

The robots are coming. Their presence will eventually bridge the digital-physical divide and dramatically impact human life by taking over tasks where our current society has shortcomings (e.g., search and rescue, elderly care, and child education). Human-centered robotics (HCR) is a vision to address how robots can coexist with humans and help people live safer, simpler and more independent lives.

As humans, we have a remarkable ability to perceive the world around us, perceive people, and interpret their behaviors. Endowing robots with these critical capabilities in highly dynamic human social environments is a significant but very challenging problem in practical …

Moving Object Detection For Interception By A Humanoid Robot, Saltanat B. Tazhibayeva

Open Access Theses

Interception of a moving object with an autonomous robot is an important problem in robotics. It has various application areas, such as in an industrial setting where products on a conveyor would be picked up by a robotic arm, in the military to halt intruders, in robotic soccer (where the robots try to get to the moving ball and try to block an opponent's attempt to pass the ball), and in other challenging situations. Interception, in and of itself, is a complex task that demands a system with target recognition capability, proper navigation and actuation toward the moving target. There …

Object Detection And Recognition In Natural Settings, George William Dittmar

Dissertations and Theses

Much research as of late has focused on biologically inspired vision models that are based on our understanding of how the visual cortex processes information. One prominent example of such a system is HMAX [17]. HMAX attempts to simulate the biological process for object recognition in cortex based on the model proposed by Hubel & Wiesel [10]. This thesis investigates the ability of an HMAX-like system (GLIMPSE [20]) to perform object-detection in cluttered natural scenes. I evaluate these results using the StreetScenes database from MIT [1, 8]. This thesis addresses three questions: (1) Can the GLIMPSE-based object detection system replicate …

Interpreting Individual Classifications Of Hierarchical Networks, Will Landecker, Michael David Thomure, Luis M.A. Bettencourt, Melanie Mitchell, Garrett T. Kenyon, Steven P. Brumby

Computer Science Faculty Publications and Presentations

Hierarchical networks are known to achieve high classification accuracy on difficult machine-learning tasks. For many applications, a clear explanation of why the data was classified a certain way is just as important as the classification itself. However, the complexity of hierarchical networks makes them ill-suited for existing explanation methods. We propose a new method, contribution propagation, that gives per-instance explanations of a trained network's classifications. We give theoretical foundations for the proposed method, and evaluate its correctness empirically. Finally, we use the resulting explanations to reveal unexpected behavior of networks that achieve high accuracy on visual object-recognition tasks using well-known …

Object Detection And Classification With Applications To Skin Cancer Screening, Jonathan Blackledge, Dmitryi Dubovitskiy

Articles

This paper discusses a new approach to the processes of object detection, recognition and classification in a digital image. The classification method is based on the application of a set of features which include fractal parameters such as the Lacunarity and Fractal Dimension. Thus, the approach used, incorporates the characterisation of an object in terms of its texture.

The principal issues associated with object recognition are presented which includes two novel fast segmentation algorithms for which C++ code is provided. The self-learning procedure for designing a decision making engine using fuzzy logic and membership function theory is also presented and …

Using Computer Vision To Create A 3d Representation Of A Snooker Table For Televised Competition Broadcasting, Hao Guo, Brian Mac Namee

Conference papers

The Snooker Extraction and 3D Builder (SE3DB) is designed to be used as a viewer aid in televised snooker broadcasting. Using a single camera positioned over a snooker table, the system creates a virtual 3D model of the table which can be used to allow audiences view the table from any angle. This would be particularly useful in allowing viewers to determine if particular shots are possible or not. This paper will describe the design, development and evaluation of this system. Particular focus in the paper will be given to the techniques used to recognise and locate the balls on …

Real Time Texture Analysis From The Parallel Computation Of Fractal Dimension, Halford I. Hayes Jr.

Computer Science Theses & Dissertations

The discrimination of texture features in an image has many important applications: from detection of man-made objects from a surrounding natural background to identification of cancerous from healthy tissue in X-ray imagery. The fractal structure in an image has been used with success to identify these features but requires unacceptable processing time if executed sequentially.

The paradigm of data parallelism is presented as the best method for applying massively parallel processing to the computation of fractal dimension of an image. With this methodology, and sufficient numbers of processors, this computation can reach real time speeds necessary for many applications. A …