Artificial Intelligence and Robotics Commons^™

Towards Dynamic Context Detection From Voice Commands And Conversations With Smart Assistants In Smart Homes, Jeniya Sultana

Graduate Theses/Dissertations

Voice-enabled interactions have become increasingly popular with the rise of voice assistants. Identifying contexts or meanings from voice commands and conversations with smart assistants can contribute to the autonomous control of smart home devices and appliances. To improve automation, there is a growing need for efficient context detection that eliminates the need to memorize voice commands. To address this need, I followed a two-step approach in my research. In the first step, I developed a unique context recognition model using a transformer, an attention mechanism, and a fully connected neural network. I trained this model on a conversational dataset of …

Flexible Attenuation Fields: Tomographic Reconstruction From Heterogeneous Datasets, Clifford S. Parker

Theses and Dissertations--Computer Science

Traditional reconstruction methods for X-ray computed tomography (CT) are highly constrained in the variety of input datasets they admit. Many of the imaging settings -- the incident energy, field-of-view, effective resolution -- remain fixed across projection images, and the only real variance is in the detector's position and orientation with respect to the scene. In contrast, methods for 3D reconstruction of natural scenes are extremely flexible to the geometric and photometric properties of the input datasets, readily accepting and benefiting from images captured under varying lighting conditions, with different cameras, and at disparate points in time and space. Extending CT …

‘I Know It When I See It’– Developing Quality Schedules Considering Subjective Or Unspecified Criteria, Douglas L. Moody

Publications and Research

Most timetabling problems have a given objective function to measure the quality of a solution. However, users may have a “I know it when I see it” recognition of a quality schedule, without specifying the complete basis for their judgment. In this situation, the objective function cannot be exclusively used as a solution quality measurement. This work presents an AI based approach to aid in categorizing the solution’s quality when the users have not explicitly defined all factors used in their criteria.

Programming By Voice, Sadia Nowrin

Dissertations, Master's Theses and Master's Reports

Programmers typically rely on a keyboard and mouse for input, which poses significant challenges for individuals with motor impairments, limiting their ability
to effectively input programs. Voice-based programming offers a promising alternative,
enabling a more inclusive and accessible programming environment. Insights from interviews with motor-impaired programmers revealed that memorizing unnatural commands in existing voice-based programming systems led to frustration. In this work, we explore how programmers naturally speak a single line of code and present a comprehensive methodology for a voice programming system aimed at making programming more accessible for diverse users. To achieve this, we adopted a two-step pipeline. …

Tension Control And Interproximation Techniques Forshape Design And Rgb-Depth Segmentation Reconstruction And Modeling, Anastasia Kazadi

Theses and Dissertations--Computer Science

Human eyes possess remarkable capabilities to perceive and interpret a wealth of information about our environment; from discerning colors and depths to identifying object boundaries and navigating obstacles, our eyes serve as invaluable guides in our daily lives. Ongoing research in the fields of computer vision and computer graphics continuously explore the ways to replicate extraordinary human vision abilities in order to develop systems and frameworks which would enable computers to capture, analyze, and act upon discerned information. In this context, this dissertation seeks to investigate and automate various shape control and data processing techniques for 3D modeling and shape …

Human-Centered Machine Learning With Interpretable Visual Knowledge Discovery, Lincoln Huber

All Master's Theses

This research advances interpretable machine learning (ML) by introducing hyperblocks (HBs) as a structured, rule-based approach for creating transparent and accurate models using meaningful numeric attributes directly interpretable to end users. Key techniques, including Parallel Hyperblock Creation, Interactive Hyperblock Creation, Level n Hyperblock Creation, and k-Nearest Neighbor Hyperblock, provide a framework that ensures domain experts can meaningfully engage with the model’s decision-making process through lossless visualizations using General Line Coordinates (GLC). Case studies with the Wisconsin Breast Cancer and MNIST datasets demonstrated HBs' effectiveness in handling high-risk and complex classification tasks, offering interpretable accuracy that traditional models struggle to achieve. …

Poster, Performed: Understanding Public Opinions Of Authorship In Generative Artificial Intelligence Models Via Analogy, Wylie Z. Kasai

Dartmouth College Master’s Theses

Over the last decade, generative artificial intelligence models have advanced significantly and provided the public with several tools to create new works of art. However, the true authorship of these works has been debated due to their training on web-scraped data. Serving as an analogy to these larger models, Poster, Performed is an interactive artificial intelligence exhibition project that uses image assets submitted by the public to create poster compositions with custom image processing algorithms. During the course of a four-day exhibition, visitors were asked to identify the exhibition’s primary artist from five options: (1) participants who submitted image assets, …

Bias-Aware Gaze Uniformity Assessment In Group Images, Omkar Kulkarni

Electronic Theses & Dissertations (2024 - present)

Today, more than 5 billion photos are captured every day, with smartphones generating over 94\% of these images. However, despite advancements in technology, achieving aesthetically pleasing group photos remains challenging, especially when it comes to aligning the direction of everyone’s gaze. While current methods focus on facial features, they often fail to ensure consistent gaze direction. The introduction of the iPhone's Live mode, which captures a 1.5-second video snippet along with still images, complicates the selection of the best key photo due to its subjective nature and a lack of publicly available data, especially during the pandemic.

To address these …

Reducing Food Scarcity: The Benefits Of Urban Farming, S.A. Claudell, Emilio Mejia

Journal of Nonprofit Innovation

Urban farming can enhance the lives of communities and help reduce food scarcity. This paper presents a conceptual prototype of an efficient urban farming community that can be scaled for a single apartment building or an entire community across all global geoeconomics regions, including densely populated cities and rural, developing towns and communities. When deployed in coordination with smart crop choices, local farm support, and efficient transportation then the result isn’t just sustainability, but also increasing fresh produce accessibility, optimizing nutritional value, eliminating the use of ‘forever chemicals’, reducing transportation costs, and fostering global environmental benefits.

Imagine Doris, who is …

Clueless: Revolutionizing Sustainable Fashion And Combating Overconsumption, Tanya Ravichandran

Graphic Communication

“Clueless” revolutionizes sustainable fashion by combating wardrobe overconsumption and the industry’s carbon footprint, using AI to suggest personalized outfits from existing wardrobes tailored to weather and wear history. It enhances user engagement through features like outfit ‘shuffle’ and provides insights into wardrobe utilization and carbon impact.

It’s more than an app; it’s a step towards a greener wardrobe and a healthier planet.

Leveraging Artificial Intelligence For Team Cognition In Human-Ai Teams, Beau Schelble

All Dissertations

Advances in artificial intelligence (AI) technologies have enabled AI to be applied across a wide variety of new fields like cryptography, art, and data analysis. Several of these fields are social in nature, including decision-making and teaming, which introduces a new set of challenges for AI research. While each of these fields has its unique challenges, the area of human-AI teaming is beset with many that center around the expectations and abilities of AI teammates. One such challenge is understanding team cognition in these human-AI teams and AI teammates' ability to contribute towards, support, and encourage it. Team cognition is …

Data-Centric Image Super-Resolution In Magnetic Resonance Imaging: Challenges And Opportunities, Mamata Shrestha

Graduate Theses and Dissertations

Super-resolution has emerged as a crucial research topic in the field of Magnetic Resonance Imaging (MRI) where it plays an important role in understanding and analysis of complex, qualitative, and quantitative characteristics of tissues at high resolutions. Deep learning techniques have been successful in achieving state-of-the-art results for super-resolution. These deep learning-based methods heavily rely on a substantial amount of data. Additionally, they require a pair of low-resolution and high-resolution images for supervised training which is often unavailable. Particularly in MRI super-resolution, it is often impossible to have low-resolution and high-resolution training image pairs. To overcome this, existing methods for …

Improving Human-Automation Collaboration In Motion Planning, Torin J. Adamson

Computer Science ETDs

Human-automation collaboration is becoming a part of everyday life as AI helps us drive, make decisions, and solve a variety of other tasks. However, safe and effective collaboration systems depend on factors in trust, communication, and more. Existing studies to explore these are typically carried out in laboratory settings, providing robust data under tight environmental control. However, human behavior evolves over time, driven by external factors that cannot be fully captured in single participation sessions. These factors form the "human context", contextualizing the behavioral data for a more complete understanding. In this thesis, video game adaptations upon conventional subject studies …

Evocative And Provocative Image-Making In The Age Of Generative Ai, Julian Kilker

Tradition Innovations in Arts, Design, and Media Higher Education

Editorial for inaugural AI-focused special issue of Tradition-Innovations in Arts, Design, and Media Higher Education, published under the auspices of the Alliance for the Arts in Research Universities (a2ru). Discusses three articles by five authors in this issue: (1) Choreographing Shadows: Interdisciplinary collaboration to orchestrate ethical image-making by Mark Burchick and Diana Pasulka; (2) Giving Up Control: Hybrid AI-augmented workflows for image-making by Joshua Vermillion; and (3) Hands are Hard: Unlearning how we talk about machine learning in the arts by Adam Hyland and Oscar Keyes.

Editing this special issue explored several key questions: What does “innovation” mean when …

Unsupervised Anomaly Detection In Medical Images With A Memory-Augmented Multi-Level Cross-Attentional Masked Autoencoder, Yu Tian, Guansong Pang, Yuyuan Liu, Chong Wang, Yuanhong Chen, Fengbei Liu, Rajvinder Singh, Johan W. Verjans, Mengyu Wang, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) aims to find anomalous images by optimising a detector using a training set that contains only normal images. UAD approaches can be based on reconstruction methods, self-supervised approaches, and Imagenet pre-trained models. Reconstruction methods, which detect anomalies from image reconstruction errors, are advantageous because they do not rely on the design of problem-specific pretext tasks needed by self-supervised approaches, and on the unreliable translation of models pre-trained from non-medical datasets. However, reconstruction methods may fail because they can have low reconstruction errors even for anomalous images. In this paper, we introduce a new reconstruction-based UAD approach …

Feature Prediction Diffusion Model For Video Anomaly Detection, Cheng Yan, Shiyu Zhang, Yang Liu, Guansong Pang, Wenjun Wang

Research Collection School Of Computing and Information Systems

Anomaly detection in the video is an important research area and a challenging task in real applications. Due to the unavailability of large-scale annotated anomaly events, most existing video anomaly detection (VAD) methods focus on learning the distribution of normal samples to detect the substantially deviated samples as anomalies. To well learn the distribution of normal motion and appearance, many auxiliary networks are employed to extract foreground object or action information. These high-level semantic features effectively filter the noise from the background to decrease its influence on detection models. However, the capability of these extra semantic models heavily affects the …

Exploring Approaches To Engage K-12 Students In Learning Computational Thinking Using Collaborative Robots, Zoila Anuri Kanu

College of Engineering Summer Undergraduate Research Program

Minority students are largely underrepresented in the STEM field. The goal for this project was to develop a program which promotes the inclusion of computation skills among students and help them work collaboratively with the use of human – robot interaction. Robots are such a strong tool that can be used to enhance computational thinking and engage students towards a technical field. Through workshops and readings about computational thinking we worked on building a block-based program that introduces the uses of robots as teaching tool for computational thinking.

Balanced Blended Space: Proposing A Universal Theoretical Framework For Combinative Reality, David Smith, Frederick Bianchi

Publications and Research

In today's fragmented societies, a unified framework for communication and collaboration across different realities is crucial. We introduce Balanced Blended Space (BBS) as a framework for describing combinative reality, encompassing virtual, physical, and conceptual realms, all intrinsically connected. Interactions within these environments shape our perceptual space. This paper outlines key axiomatic assumptions, criteria for a universal framework, and fundamental terminology. We identify deep symmetries enabling the BBS framework, including Cognitive and Computational Symmetry, Physical and Virtual Symmetry, Mediation Pathway Symmetry, Space-Time Symmetry, and Sensory Symmetry. We propose tests to determine its viability, emphasizing virtual intelligence as a collaborative partner. We …

Flood: A Flexible Invariant Learning Framework For Out-Of-Distribution Generalization On Graphs, Yang Liu, Xiang Ao, Fuli Feng, Yunshan Ma, Kuan Li, Tat‑Seng Chua, Qing He

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) have achieved remarkable success in various domains but most of them are developed under the in-distribution assumption. Under out-of-distribution (OOD) settings, they suffer from the distribution shift between the training set and the test set and may not generalize well to the test distribution. Several methods have tried the invariance principle to improve the generalization of GNNs in OOD settings. However, in previous solutions, the graph encoder is immutable after the invariant learning and cannot be adapted to the target distribution flexibly. Confronting the distribution shift, a flexible encoder with refinement to the target distribution can …

Balanced Blended Space: Foundational Human–Ai Dialogues In A Symmetry-Based Mediation Framework, David Smith

Publications and Research

This working paper documents the early development of the Balanced Blended Space (BBS) framework through a series of iterative interactions between a cognitive agent (human researcher) and a computational agent (AI system) conducted in 2023. The work is motivated by the need for a universal theoretical model capable of describing the integration of physical, virtual, and conceptual spaces, particularly in response to increasing fragmentation across contemporary communication systems.

BBS is proposed as a symmetry-based mediation framework in which relationships between domains—such as physical and virtual space, cognition and computation, and multiple sensory modalities—are treated as structurally equivalent and mappable. Central …

Fine-Grained Domain Adaptive Crowd Counting Via Point-Derived Segmentation, Yongtuo Liu, Dan Xu, Sucheng Ren, Hanjie Wu, Hongmin Cai, Shengfeng He

Research Collection School Of Computing and Information Systems

Due to domain shift, a large performance drop is usually observed when a trained crowd counting model is deployed in the wild. While existing domain-adaptive crowd counting methods achieve promising results, they typically regard each crowd image as a whole and reduce domain discrepancies in a holistic manner, thus limiting further improvement of domain adaptation performance. To this end, we propose to untangle domain-invariant crowd and domain-specific background from crowd images and design a fine-grained domain adaption method for crowd counting. Specifically, to disentangle crowd from background, we propose to learn crowd segmentation from point-level crowd counting annotations in a …

An Investigation Into Machine Learning Techniques For Designing Dynamic Difficulty Agents In Real-Time Games, Ryan Adare Dunagan

Electronic Theses and Dissertations

Video games are an incredibly popular pastime enjoyed by people of all ages world wide. Many different kinds of games exist, but most games feature some elements of the player overcoming some challenge, usually through gameplay. These challenges are insurmountable for some people and may turn them off to video games as a pastime. Games can be made more accessible to players of little skill and/or experience through the use of Dynamic Difficulty Adjustment (DDA) systems that adjust the difficulty of the game in response to the player’s performance. This research seeks to establish the effectiveness of machine learning techniques …

Curricular Contrastive Regularization For Physics-Aware Single Image Dehazing, Yu Zheng, Jiahui Zhan, Shengfeng He, Yong Du

Research Collection School Of Computing and Information Systems

Considering the ill-posed nature, contrastive regularization has been developed for single image dehazing, introducing the information from negative images as a lower bound. However, the contrastive samples are non-consensual, as the negatives are usually represented distantly from the clear (i.e., positive) image, leaving the solution space still under-constricted. Moreover, the interpretability of deep dehazing models is underexplored towards the physics of the hazing process. In this paper, we propose a novel curricular contrastive regularization targeted at a consensual contrastive space as opposed to a non-consensual one. Our negatives, which provide better lower-bound constraints, can be assembled from 1) the hazy …

Venus: A Geometrical Representation For Quantum State Visualization, Shaolun Ruan, Ribo Yuan, Qiang Guan, Yanna Lin, Ying Mao, Weiwen Jiang, Zhepeng Wang, Wei Xu, Yong Wang

Research Collection School Of Computing and Information Systems

Visualizations have played a crucial role in helping quantum computing users explore quantum states in various quantum computing applications. Among them, Bloch Sphere is the widely-used visualization for showing quantum states, which leverages angles to represent quantum amplitudes. However, it cannot support the visualization of quantum entanglement and superposition, the two essential properties of quantum computing. To address this issue, we propose VENUS, a novel visualization for quantum state representation. By explicitly correlating 2D geometric shapes based on the math foundation of quantum computing characteristics, VENUS effectively represents quantum amplitudes of both the single qubit and two qubits for quantum …

Mosaic: Spatially-Multiplexed Edge Ai Optimization Over Multiple Concurrent Video Sensing Streams, Ila Gokarn, Hemanth Sabbella, Yigong Hu, Tarek Abdelzaher, Archan Misra

Research Collection School Of Computing and Information Systems

Sustaining high fidelity and high throughput of perception tasks over vision sensor streams on edge devices remains a formidable challenge, especially given the continuing increase in image sizes (e.g., generated by 4K cameras) and complexity of DNN models. One promising approach involves criticality-aware processing, where the computation is directed selectively to "critical" portions of individual image frames. We introduce MOSAIC, a novel system for such criticality-aware concurrent processing of multiple vision sensing streams that provides a multiplicative increase in the achievable throughput with negligible loss in perception fidelity. MOSAIC determines critical regions from images received from multiple vision …

Chronos: Time-Aware Zero-Shot Identification Of Libraries From Vulnerability Reports, Yunbo Lyu, Thanh Le Cong, Hong Jin Kang, Ratnadira Widyasari, Zhipeng Zhao, Xuan-Bach Dinh Le, Ming Li, David Lo

Research Collection School Of Computing and Information Systems

Tools that alert developers about library vulnerabilities depend on accurate, up-to-date vulnerability databases which are maintained by security researchers. These databases record the libraries related to each vulnerability. However, the vulnerability reports may not explicitly list every library and human analysis is required to determine all the relevant libraries. Human analysis may be slow and expensive, which motivates the need for automated approaches. Researchers and practitioners have proposed to automatically identify libraries from vulnerability reports using extreme multi-label learning (XML). While state-of-the-art XML techniques showed promising performance, their experimental settings do not practically fit what happens in reality. Previous studies …

Artificial: A Study On The Use Of Artificial Intelligence In Art, Hayden Ernst

Theses/Capstones/Creative Projects

In the past three to five years there have been significant improvements made in AI due to improvements in computing capacity, the collection and use of big data, and an increase in public interest and funding for research. Programs such as ChatGPT, DALL•E, and Midjourney have also gained tremendous popularity in a relatively short amount of time. This led me to this project in which I aimed to gain a deeper understanding of these art generator AI and where they fit into art as a whole. My goal was to give recommendations to museums and exhibits in Omaha on what …

Open-Set Domain Adaptation By Deconfounding Domain Gaps, Xin Zhao, Shengsheng Wang, Qianru Sun

Research Collection School Of Computing and Information Systems

Open-Set Domain Adaptation (OSDA) aims to adapt the model trained on a source domain to the recognition tasks in a target domain while shielding any distractions caused by open-set classes, i.e., the classes “unknown” to the source model. Compared to standard DA, the key of OSDA lies in the separation between known and unknown classes. Existing OSDA methods often fail the separation because of overlooking the confounders (i.e., the domain gaps), which means their recognition of “unknown classes” is not because of class semantics but domain difference (e.g., styles and contexts). We address this issue by explicitly deconfounding domain gaps …

Supporting Novices Author Audio Descriptions Via Automatic Feedback, Rosiana Natalie, Joshua Shi-Hao Tseng, Hernisa Kacorri, Kotaro Hara

Research Collection School Of Computing and Information Systems

Audio descriptions (AD) make videos accessible to those who cannot see them. But many videos lack AD and remain inaccessible as traditional approaches involve expensive professional production. We aim to lower production costs by involving novices in this process. We present an AD authoring system that supports novices to write scene descriptions (SD)—textual descriptions of video scenes—and convert them into AD via text-to-speech. The system combines video scene recognition and natural language processing to review novice-written SD and feeds back what to mention automatically. To assess the effectiveness of this automatic feedback in supporting novices, we recruited 60 participants to …

Chatgpt As Metamorphosis Designer For The Future Of Artificial Intelligence (Ai): A Conceptual Investigation, Amarjit Kumar Singh (Library Assistant), Dr. Pankaj Mathur (Deputy Librarian)

Library Philosophy and Practice (e-journal)

Abstract

Purpose: The purpose of this research paper is to explore ChatGPT’s potential as an innovative designer tool for the future development of artificial intelligence. Specifically, this conceptual investigation aims to analyze ChatGPT’s capabilities as a tool for designing and developing near about human intelligent systems for futuristic used and developed in the field of Artificial Intelligence (AI). Also with the helps of this paper, researchers are analyzed the strengths and weaknesses of ChatGPT as a tool, and identify possible areas for improvement in its development and implementation. This investigation focused on the various features and functions of ChatGPT that …