Engineering Commons^™

Reasoning Across Language And Vision In Machines And Humans, Andrei Barbu

Open Access Dissertations

Humans not only outperform AI and computer-vision systems, but use an unknown computational mechanism to perform tasks for which no suitable approaches exist. I present work investigating both novel tasks and how humans approach them in the context of computer vision and linguistics. I demonstrate a system which, like children, acquires high-level linguistic knowledge about the world. Robots learn to play physically-instantiated board games and use that knowledge to engage in physical play. To further integrate language and vision I develop an approach which produces rich sentential descriptions of events depicted in videos. I then show how to simultaneously detect …

Hypothesize-And-Verify Based Solutions For Place Recognition And Mobile Robot Self-Localization In Interior Hallways, Khalil Mustafa Ahmad Yousef

Open Access Dissertations

There is much research interest currently in having mobile robots build accurate and visually dense models ofinterior space as they traverse through such spaces. One of the interesting problems that has came out of this research is that of visual place recognition and self-localization. This is the problem that forms the focus of the present dissertation. We show how dense and accurate 3D models of the interior space can be constructed using a hierarchical sensor-fusion architecture. Our system fuses images from a single photometric camera with range data from a laser scanning sensor. The range data used is rudimentary--the range …

Semantically Grounded Learning From Unstructured Demonstrations, Scott D. Niekum

Open Access Dissertations

Robots exhibit flexible behavior largely in proportion to their degree of semantic knowledge about the world. Such knowledge is often meticulously hand-coded for a narrow class of tasks, limiting the scope of possible robot competencies. Thus, the primary limiting factor of robot capabilities is often not the physical attributes of the robot, but the limited time and skill of expert programmers. One way to deal with the vast number of situations and environments that robots face outside the laboratory is to provide users with simple methods for programming robots that do not require the skill of an expert.

For this …