Computer Sciences Commons^™

Reasoning Across Language And Vision In Machines And Humans, Andrei Barbu

Open Access Dissertations

Humans not only outperform AI and computer-vision systems, but use an unknown computational mechanism to perform tasks for which no suitable approaches exist. I present work investigating both novel tasks and how humans approach them in the context of computer vision and linguistics. I demonstrate a system which, like children, acquires high-level linguistic knowledge about the world. Robots learn to play physically-instantiated board games and use that knowledge to engage in physical play. To further integrate language and vision I develop an approach which produces rich sentential descriptions of events depicted in videos. I then show how to simultaneously detect …

Computer Sketch Recognition, Richard Steigerwald

Master's Theses

Tens of thousands of years ago, humans drew sketches that we can see and identify even today. Sketches are the oldest recorded form of human communication and are still widely used. The universality of sketches supersedes that of culture and language. Despite the universal accessibility of sketches by humans, computers are unable to interpret or even correctly identify the contents of sketches drawn by humans with a practical level of accuracy.

In my thesis, I demonstrate that the accuracy of existing sketch recognition techniques can be improved by optimizing the classification criteria. Current techniques classify a 20,000 sketch crowd-sourced dataset …

Identifying Robust Sift Features For Improved Image Alignment, Sanjay Abhinav Vemuri

Graduate Theses and Dissertations

In this thesis, we will study different ways to improve feature matching by increasing the quality and reducing the number of SIFT features. We created an algorithm to identify robust SIFT features by evaluating how invariant individual feature points are to changes in scale. This allows us to exclude poor SIFT feature points from the matching process and obtain better matching results in reduced time. We also developed techniques consider scale ratios and changes in object orientation when performing feature matching. This allows us to exclude false-positive feature matches and obtain better image alignment results.

An Architecture For Online Semantic Labeling On Ugvs, Arne Suppe, Luis Navarro-Serment, Daniel Munoz, Drew Bagnell, Martial Hebert

Research Collection School Of Computing and Information Systems

We describe an architecture to provide online semantic labeling capabilities to field robots operating in urban environments. At the core of our system is the stacked hierarchical classifier developed by Munoz et al.,1 which classifies regions in monocular color images using models derived from hand labeled training data. The classifier is trained to identify buildings, several kinds of hard surfaces, grass, trees, and sky. When taking this algorithm into the real world, practical concerns with difficult and varying lighting conditions require careful control of the imaging process. First, camera exposure is controlled by software, examining all of the image’s pixels, …

Face Tracking User Interfaces Using Vision-Based Consumer Devices, Norman Villaroman

Theses and Dissertations

Some individuals have difficulty using standard hand-manipulated input devices such as a mouse and a keyboard effectively. For such users who at the same time have sufficient control over face and head movement, a robust perceptual or vision-based user interface that can track face movement can significantly help them. Using vision-based consumer devices makes such a user interface readily available and allows its use to be non-intrusive. Designing this type of user interface presents some significant challenges particularly with accuracy and usability. This research investigates such problems and proposes solutions to create a usable and robust face tracking user interface …

Object Detection And Recognition In Natural Settings, George William Dittmar

Dissertations and Theses

Much research as of late has focused on biologically inspired vision models that are based on our understanding of how the visual cortex processes information. One prominent example of such a system is HMAX [17]. HMAX attempts to simulate the biological process for object recognition in cortex based on the model proposed by Hubel & Wiesel [10]. This thesis investigates the ability of an HMAX-like system (GLIMPSE [20]) to perform object-detection in cluttered natural scenes. I evaluate these results using the StreetScenes database from MIT [1, 8]. This thesis addresses three questions: (1) Can the GLIMPSE-based object detection system replicate …

Interpreting Individual Classifications Of Hierarchical Networks, Will Landecker, Michael David Thomure, Luis M.A. Bettencourt, Melanie Mitchell, Garrett T. Kenyon, Steven P. Brumby

Computer Science Faculty Publications and Presentations

Hierarchical networks are known to achieve high classification accuracy on difficult machine-learning tasks. For many applications, a clear explanation of why the data was classified a certain way is just as important as the classification itself. However, the complexity of hierarchical networks makes them ill-suited for existing explanation methods. We propose a new method, contribution propagation, that gives per-instance explanations of a trained network's classifications. We give theoretical foundations for the proposed method, and evaluate its correctness empirically. Finally, we use the resulting explanations to reveal unexpected behavior of networks that achieve high accuracy on visual object-recognition tasks using well-known …

On The Role Of Shape Prototypes In Hierarchical Models Of Vision, Michael David Thomure, Melanie Mitchell, Garrett T. Kenyon

Computer Science Faculty Publications and Presentations

We investigate the role of learned shape-prototypes in an influential family of hierarchical neural-network models of vision. Central to these networks’ design is a dictionary of learned shapes, which are meant to respond to discriminative visual patterns in the input. While higher-level features based on such learned prototypes have been cited as key for viewpointinvariant object-recognition in these models [1], [2], we show that high performance on invariant object-recognition tasks can be obtained by using a simple set of unlearned, “shape-free” features. This behavior is robust to the size of the network. These results call into question the roles of …

Unmanned Autonomous Object Retrieval: Old Dominion University 2013 International Aerial Robotics Competition Entry, Johnathan Bailey, Austin Boyd, Chung-Hao Chen, Stephen Dailey, Lisa Henderson, Jeremy Stuart, Christina Williams

Electrical & Computer Engineering Faculty Publications

This paper describes the design implementation of a Quadrotor Unmanned Aerial Vehicle (UAV) with the capability of exploring indoor locations without the assistance of external aids. For relative position, the use of a laser range sensor, an optical flow sensor, and sonar sensor combined allows for the vehicle to generate mapping information. With relative position in mind, the vehicle uses vision algorithms to recognize immediate obstacles, sign, and entry ways to allow for quick movement responses and object recognition. A proportional-integral-differentiator controller allows for flight stability and mitigation in the tight confines of the indoor spaces. A mapping algorithm allows …

Three-Dimensional Scene Reconstruction Using Multiple Microsoft Kinects, Matt Miller

Graduate Theses and Dissertations

The Microsoft Kinect represents a leap forward in the form of cheap, consumer friendly, depth sensing cameras. Through the use of the depth information as well as the accompanying RGB camera image, it becomes possible to represent the scene, what the camera sees, as a three-dimensional geometric model. In this thesis, we explore how to obtain useful data from the Kinect, and how to use it for the creation of a three-dimensional geometric model of the scene. We develop and test multiple ways of improving the depth information received from the Kinect, in order to create smoother three-dimensional models. We …

Gradient Based Mrf Learning For Image Restoration And Segmentation, Kegan Samuel

Electronic Theses and Dissertations

The undirected graphical model or Markov Random Field (MRF) is one of the more popular models used in computer vision and is the type of model with which this work is concerned. Models based on these methods have proven to be particularly useful in low-level vision systems and have led to state-of-the-art results for MRF-based systems. The research presented will describe a new discriminative training algorithm and its implementation. The MRF model will be trained by optimizing its parameters so that the minimum energy solution of the model is as similar as possible to the ground-truth. While previous work has …

A Study Of Localization And Latency Reduction For Action Recognition, Syed Zain Masood

Electronic Theses and Dissertations

The success of recognizing periodic actions in single-person-simple-background datasets, such as Weizmann and KTH, has created a need for more complex datasets to push the performance of action recognition systems. In this work, we create a new synthetic action dataset and use it to highlight weaknesses in current recognition systems. Experiments show that introducing background complexity to action video sequences causes a significant degradation in recognition performance. Moreover, this degradation cannot be fixed by fine-tuning system parameters or by selecting better feature points. Instead, we show that the problem lies in the spatio-temporal cuboid volume extracted from the interest point …

Object Retrieval From Secure Unknown Interior Spaces Using Autonomous Unmanned Aerial Vehicles, John Levous, Julie Hoven, Victor Habgood, Abdulrahman Alotaibi, Brandon Ordway, Garibe Mohammed-Jones, Haole Guo, Filip Cuckov, Chung-Hao Chen

Electrical & Computer Engineering Faculty Publications

This paper describes an autonomous unmanned aerial vehicle (UAV) designed to participate in the 23rd annual International Aerial Robotics Competition. The UAV is equipped with onboard sensors and a Harvard architecture 8-bit RISC microcontroller to monitor and locally control its flight telemetry. Additional sensors (and an additional microcontroller) are used for detecting and mapping of structural and environmental objects while the UAV is in flight. The microcontrollers are interfaced with wireless communication modules for transmitting flight telemetry and structural/environmental data to a ground control station that sends the UAV command and control signals required for the mission objectives. The UAV …

Toward Automatic Subpixel Registration Of Unmanned Airborne Vehicle Images, Amr Hussein Yousef, Jiang Li, Mohammad Karim, Mark Allen Neifeld (Ed.), Amit Ashok (Ed.)

Electrical & Computer Engineering Faculty Publications

Many applications require to register images within subpixel accuracy like computer vision especially super-resolution (SR) where the estimated subpixel shifts are very crucial in the reconstruction and restoration of SR images. In our work we have an optical sensor that is mounted on an unmanned airborne vehicle (UAV) and captures a set of images that contain sufficient overlapped area required to reconstruct a SR image. Due to the wind, The UAV may encounter rotational effects such as yaw, pitch and roll which can distort the acquired as well as processed images with shear, tilt or perspective distortions. In this paper …

Joint Source-Channel Coding For Error Resilient Transmission Of Static 3d Models, Mehmet Oğuz Bi̇ci̇, Andrey Norkin, Gözde Akar

Turkish Journal of Electrical Engineering and Computer Sciences

In this paper, performance analysis of joint source-channel coding techniques for error-resilient transmission of three dimensional (3D) models are presented. In particular, packet based transmission scenarios are analyzed. The packet loss resilient methods are classified into two groups according to progressive compression schemes employed: Compressed Progressive Meshes (CPM) based methods and wavelet based methods. In the first group, layers of CPM algorithm are protected unequally by Forward Error Correction (FEC) using Reed Solomon (RS) codes. In the second group, embedded bitstream obtained from wavelet based coding is protected unequally with FEC as well. Both groups of methods are scalable with …

Development Of Advanced Algorithms To Detect, Characterize And Forecast Solar Activities, Yuan Yuan

Dissertations

Study of the solar activity is an important part of space weather research. It is facing serious challenges because of large data volume, which requires application of state-of-the-art machine learning and computer vision techniques. This dissertation targets at two essential aspects in space weather research: automatic feature detection and forecasting of eruptive events.

Feature detection includes solar filament detection and solar fibril tracing. A solar filament consists of a mass of gas suspended over the chromosphere by magnetic fields and seen as a dark, ribbon-shaped feature on the bright solar disk in Hα (Hydrogen-alpha) full-disk solar images. In this dissertation, …

Learning Local Features Using Boosted Trees For Face Recognition, Rajkiran Gottumukkal

Electrical & Computer Engineering Theses & Dissertations

Face recognition is fundamental to a number of significant applications that include but not limited to video surveillance and content based image retrieval. Some of the challenges which make this task difficult are variations in faces due to changes in pose, illumination and deformation. This dissertation proposes a face recognition system to overcome these difficulties. We propose methods for different stages of face recognition which will make the system more robust to these variations. We propose a novel method to perform skin segmentation which is fast and able to perform well under different illumination conditions. We also propose a method …

Face Recognition Using Multiple Features In Different Color Spaces, Zhiming Liu

Dissertations

Face recognition as a particular problem of pattern recognition has been attracting substantial attention from researchers in computer vision, pattern recognition, and machine learning. The recent Face Recognition Grand Challenge (FRGC) program reveals that uncontrolled illumination conditions pose grand challenges to face recognition performance. Most of the existing face recognition methods use gray-scale face images, which have been shown insufficient to tackle these challenges. To overcome this challenging problem in face recognition, this dissertation applies multiple features derived from the color images instead of the intensity images only.

First, this dissertation presents two face recognition methods, which operate in different …

Enhancement Technique For Aerial Images, Sertan Erkanli, Ahmet Gungor Pakfiliz, Jiang Li

Electrical & Computer Engineering Faculty Publications

Recently, we proposed an enhancement technique for uniformly and non-uniformly illuminated dark images that provides high color accuracy and better balance between the luminance and the contrast in images to improve the visual representations of digital images. In this paper we define an improved version of the proposed algorithm to enhance aerial images in order to reduce the gap between direct observation of a scene and its recorded image.

Automated Extraction Of Structures From Sketches Of Biological Specimens, Jamie Joseph Schirf

School of Computing: Dissertations, Theses, and Student Research

The goal of this study was to develop automated techniques to extract biological structures from sketches of biological specimens. This will form the basis for a searchable database of information about the specimens. Having such a database enables researchers to efficiently search for specimens with particular qualities or identify unknown specimens. After some preprocessing of the images, the important internal organs of the specimen are extracted using image analysis techniques. The shape, size, and organization of the organs are used to categorize and then to reorganize them in the image. Results using a large database of sketches of trematodes, in …

Understanding Classification Decisions For Object Detection, Will Landecker, Michael David Thomure, Melanie Mitchell

Systems Science Friday Noon Seminar Series

Computer vision systems are traditionally tested in the object detection paradigm. In these experiments, a vision system is asked whether or not a specific object--for example an animal--occurs in a given image. A system that often answers correctly is said to be very accurate. In this talk, we will discuss some ambiguity that exists in this measure of accuracy. We will also propose a new measure of object-detection accuracy that addresses some of this ambiguity, and apply this measure to the hierarchical "standard model" of visual cortex.

Markerless Tracking Using Polar Correlation Of Camera Optical Flow, Prince Gupta

Electronic Theses and Dissertations

We present a novel, real-time, markerless vision-based tracking system, employing a rigid orthogonal configuration of two pairs of opposing cameras. Our system uses optical flow over sparse features to overcome the limitation of vision-based systems that require markers or a pre-loaded model of the physical environment. We show how opposing cameras enable cancellation of common components of optical flow leading to an efficient tracking algorithm that captures five degrees of freedom including direction of translation and angular velocity. Experiments comparing our device with an electromagnetic tracker show that its average tracking accuracy is 80% over 185 frames, and it is …

Spatio-Temporal Maximum Average Correlation Height Templates In Action Recognition And Video Summarization, Mikel Rodriguez

Electronic Theses and Dissertations

Action recognition represents one of the most difficult problems in computer vision given that it embodies the combination of several uncertain attributes, such as the subtle variability associated with individual human behavior and the challenges that come with viewpoint variations, scale changes and different temporal extents. Nevertheless, action recognition solutions are critical in a great number of domains, such video surveillance, assisted living environments, video search, interfaces, and virtual reality. In this dissertation, we investigate template-based action recognition algorithms that can incorporate the information contained in a set of training examples, and we explore how these algorithms perform in action …

An Optical Machine Vision System For Applications In Cytopathology, Jonathan Blackledge, Dmitry Dubovitskiy

Articles

This paper discusses a new approach to the processes of object detection, recognition and classification in a digital image focusing on problem in Cytopathology. A unique self learning procedure is presented in order to incorporate expert knowledge. The classification method is based on the application of a set of features which includes fractal parameters such as the Lacunarity and Fourier dimension. Thus, the approach includes the characterisation of an object in terms of its fractal properties and texture characteristics. The principal issues associated with object recognition are presented which include the basic model and segmentation algorithms. The self-learning procedure for …

Accurate Tracking Of Objects Using Level Sets, Nalin Senthamil

All Theses

Our current work presents an approach to tackle the challenging task of tracking objects in Internet videos taken from large web repositories such as YouTube. Such videos more often than not, are captured by users using their personal hand-held cameras and cellphones and hence suffer from problems such as poor quality, camera jitter and unconstrained lighting and environmental settings. Also, it has been observed that events being recorded by such videos usually contain objects moving in an unconstrained fashion. Hence, tracking objects in Internet videos is a very challenging task in the field of computer vision since there is no …

Automatic Detection Of Child Pornography, Glen Thompson

Australian Digital Forensics Conference

Before the introduction of the internet, the availability of child pornography was reported as on the decline (Jenkins 2001). Since its emergence, however, the internet has made child pornography a much more accessible and available means of trafficking across borders (Biegel 2001; Jenkins 2001; Wells, Finkelhor et al. 2007). The internet as it is at present is made up of a vast array of protocols and networks where traffickers can anonymously share large volumes of illegal material amongst each other from locations with relaxed or non-existent laws that prohibit the possession or trafficking of illegal material. Likewise the internet is …

Using Predictive Rendering As A Vision-Aided Technique For Autonomous Aerial Refueling, Adam D. Weaver

Theses and Dissertations

This research effort seeks to characterize a vision-aided approach for an Unmanned Aerial System (UAS) to autonomously determine relative position to another aircraft in a formation, specifically to address the autonomous aerial refueling problem. A system consisting of a monocular digital camera coupled with inertial sensors onboard the UAS is analyzed for feasibility of using this vision-aided approach. A three-dimensional rendering of the tanker aircraft is used to generate predicted images of the tanker as seen by the receiver aircraft. A rigorous error model is developed to model the relative dynamics between an INS-equipped receiver and the tanker aircraft. A …

Real-Time Image-Based Motion Detection Using Color And Structure, Manali Chakraborty

Open Access Theses & Dissertations

Motion detection is an important problem in computer vision and has multiple applications in the real world, including surveillance-related activities, gaming, and human-robot interactions. These systems need to be robust enough to handle fluctuations in light intensity and other external factors like noise and compression artifacts. In this thesis a method is proposed for detecting the regions of motion from a video sequence in real time. The main idea of this work is to detect motion based on both structure and color. Structure-based detection is carried out using information from the Census Transform computed on gradient images obtained with Sobel …

A Subspace Projection Methodology For Nonlinear Manifold Based Face Recognition, Praveen Sankaran

Electrical & Computer Engineering Theses & Dissertations

A novel feature extraction method that utilizes nonlinear mapping from the original data space to the feature space is presented in this dissertation. Feature extraction methods aim to find compact representations of data that are easy to classify. Measurements with similar values are grouped to same category, while those with differing values are deemed to be of separate categories. For most practical systems, the meaningful features of a pattern class lie in a low dimensional nonlinear constraint region (manifold) within the high dimensional data space. A learning algorithm to model this nonlinear region and to project patterns to this feature …

A See-Ability Metric To Improve Mini Unmanned Aerial Vehicle Operator Awareness Using Video Georegistered To Terrain Models, Cameron Howard Engh

Theses and Dissertations

Search and rescue operations conducted in wilderness environments can be greatly aided by the use of video filmed from mini-UAVs. While lightweight, inexpensive and easily transportable, these small aircraft suffer from wind buffeting and may produce video that is difficult to search. To aid in the video search process, we have created a system to project video frames into a 3D representation of the search region. This projection allows us to tie each frame of video to a real-world location, enabling a myriad of novel views, mosaics and metrics that can be used to guide the search including a new …