Graphics and Human Computer Interfaces Commons^™

Sound In Color, Amber Rhodes

Honors Scholars Collaborative Projects

“Sound in Color” is an interactive audio-visual experience designed to explore the relationship between sound, color, and emotions. Taking place on the Massey Concert Hall stage, the project is inspired by synesthesia and incorporates research on color psychology. Participants are invited to select an emotion and color. As the user hums into a microphone, they hear their emotions expressed through sound in their headphones and watch as the lights on stage respond to their vocal cues.

Privacyprimer: Towards Privacy-Preserving Episodic Memory Support For Older Adults, Thivya Kandappu, Vigneshwaran Subbaraju, Qianli Xu

Research Collection School Of Computing and Information Systems

Built-in pervasive cameras have become an integral part of mobile/wearable devices and enabled a wide range of ubiquitous applications with their ability to be "always-on". In particular, life-logging has been identified as a means to enhance the quality of life of older adults by allowing them to reminisce about their own life experiences. However, the sensitive images captured by the cameras threaten individuals' right to have private social lives and raise concerns about privacy and security in the physical world. This threat gets worse when image recognition technologies can link images to people, scenes, and objects, hence, implicitly and unexpectedly …

Learning And Simulation Algorithms For Constraint Physical Systems, Shuqi Yang

Dartmouth College Master’s Theses

This thesis explores two computational approaches to learn and simulate complex physical systems exhibiting constraint characteristics. The target applications encompass both solids and fluids. On the solid side, we proposed a new family of data-driven simulators to predict the behaviors of an unknown physical system by learning its underpinning constraints. We devised a neural projection operator facilitated by an embedded recursive neural network to interactively enforce the learned underpinning constraints and to predict its various physical behaviors. Our method can automatically uncover a broad range of constraints from observation point data, such as length, angle, bending, collision, boundary effects, and …

Recent Advances In Smartphone Computational Photography, Paul Friederichsen

Scholarly Horizons: University of Minnesota, Morris Undergraduate Journal

Smartphone cameras present many challenges, most of which come from the need for them to be physically small. Their small size puts a fundamental limit on their ability to resolve detail and collect light, which makes low-light photography and zooming difficult. This paper presents two approaches to improve smartphone photography through software techniques. The first is handheld super-resolution which uses natural hand movement to improve the resolution smartphone images, especially when zoomed. The second approach is a system which improves low light photography in smartphones.

Physics Engine On The Gpu With Opengl Compute Shaders, Quan Huy Minh Bui

Master's Theses

Any kind of graphics simulation can be thought of like a fancy flipbook. This notion is, of course, nothing new. For instance, in a game, the central computing unit (CPU) needs to process frame by frame, figuring out what is happening, and then finally issues draw calls to the graphics processing unit (GPU) to render the frame and display it onto the monitor. Traditionally, the CPU has to process a lot of things: from the creation of the window environment for the processed frames to be displayed, handling game logic, processing artificial intelligence (AI) for non-player characters (NPC), to the …

Gpu High-Performance Framework For Pic-Like Simulation Methods Using The Vulkan® Explicit Api, Kolton Jacob Yager

Master's Theses

Within computational continuum mechanics there exists a large category of simulation methods which operate by tracking Lagrangian particles over an Eulerian background grid. These Lagrangian/Eulerian hybrid methods, descendants of the Particle-In-Cell method (PIC), have proven highly effective at simulating a broad range of materials and mechanics including fluids, solids, granular materials, and plasma. These methods remain an area of active research after several decades, and their applications can be found across scientific, engineering, and entertainment disciplines.

This thesis presents a GPU driven PIC-like simulation framework created using the Vulkan® API. Vulkan is a cross-platform and open-standard explicit API for graphics …

3d Printing Of Human Microbiome Constituents To Understand Spatial Relationships And Shape Parameters In Bacteriology, Jacques Izard, Teklu Kuru Gerbaba, Shara R. P. Yumul

Department of Food Science and Technology: Faculty Publications

Effective laboratory and classroom demonstration of microbiome size and shape, diversity, and ecological relationships is hampered by a lack of high-resolution, easy-to-use, readily accessible physical or digital models for use in teaching. Three-dimensional (3D) representations are, overall, more effective in communicating visuospatial information, allowing for a better understanding of concepts not directly observable with the unaided eye. Published morphology descriptions and microscopy images were used as the basis for designing 3D digital models, scaled at 20,000×, using computer-aided design software (CAD) and generating printed models of bacteria on mass-market 3D printers. Sixteen models are presented, including rod-shaped, spiral, flask-like, vibroid, …

Display Design To Avoid And Mitigate Limit Cycle Oscillations (Lco) On The F-16c, David J. Feibus

Theses and Dissertations

The U.S. Air Force F-16 Fighting Falcons flying characteristics and flight envelope are dynamic and defined by its external weapon stores configuration. The employment of its munitions at certain speeds can put the F-16 into a flutter-like state in which Limit Cycle Oscillations (LCO) are induced. In LCO, a pilots fine motor control might be hindered, and the aircraft may lose combat effectiveness until flight conditions are reduced. The current research attempted to provide pilots with a predictive feedback display to avoid an LCO-susceptible configuration by increasing their situation awareness about the consequences of employing certain munitions to their flight …

Google Books, Jody Condit Fagan

Libraries

Google Books’ (GB) full-text search of more than 40 million books offers significant value for libraries and their patrons. However, Google’s refusal to disclose information about the coverage of GB, as well as observed gaps and inaccuracies in the collection and its metadata, makes it difficult to recommend with confidence for a given research need. While most search and retrieval functions work well, glitches aren’t hard to find, which suggests GB development is focused on user experiences that relate to monetization. Privacy and equity concerns surrounding GB mirror those of other big technology platforms. Still, every librarian should familiarize themselves …

Deep Learning-Based Breast Cancer Detection And Classification Using Histopathology Images, Ghulam Murtaza

Student Works (2020-2029)

Cancer disease is drastically increasing worldwide over the past few years. Among all types of cancers in women, breast cancer (BrC) is the main cause of abnormal deaths. For a confident diagnosis of BrC, histopathology (Hp) images are usually suggested by the doctors. BrC detection is a diagnostic test for benign (non-cancerous) and malignant (cancerous) breast tumors (BrT). Once the BrT is diagnosed, then it needs to be classified for subtypes of benign and malignant to start specific treatment. Several studies developed BrC detection and classification models using Hp images. However, the existing models required high computational resources, long training …

Olympic Games Event Recognition Via Transfer Learning With Photobombing Guided Data Augmentation, Yousef I. Mohamad, Samah S. Baraheem, Tam Van Nguyen

Computer Science Faculty Publications

Automatic event recognition in sports photos is both an interesting and valuable research topic in the field of computer vision and deep learning. With the rapid increase and the explosive spread of data, which is being captured momentarily, the need for fast and precise access to the right information has become a challenging task with considerable importance for multiple practical applications, i.e., sports image and video search, sport data analysis, healthcare monitoring applications, monitoring and surveillance systems for indoor and outdoor activities, and video captioning. In this paper, we evaluate different deep learning models in recognizing and interpreting the sport …

Evoking Empathy: A Framework For Describing Empathy Tools, Sydney Pratte, Anthony Tang, Lora Oehlberg

Research Collection School Of Computing and Information Systems

Empathy tools are experiences designed to evoke empathetic responses by placing the user in another’s lived and felt experience. The problem is that designers do not have a common vocabulary to describe empathy tool experiences; consequently, it is difficult to compare/contrast empathy tool designs or to think about their efficacy. To address this problem, we analyzed 26 publications on empathy tools to develop a descriptive framework for designers of empathy tools. Based on our analysis, we found that empathy tools can be described along three dimensions: (i) the amount of agency the tool allows, (ii) the user’s perspective while using …

Adversarial Meta Sampling For Multilingual Low-Resource Speech Recognition, Yubei Xiao, Ke Gong, Pan Zhou, Guolin Zheng, Xiaodan Liang, Liang Lin

Research Collection School Of Computing and Information Systems

Human doctors with well-structured medical knowledge can diagnose a disease merely via a few conversations with patients about symptoms. In contrast, existing knowledgegrounded dialogue systems often require a large number of dialogue instances to learn as they fail to capture the correlations between different diseases and neglect the diagnostic experience shared among them. To address this issue, we propose a more natural and practical paradigm, i.e., low-resource medical dialogue generation, which can transfer the diagnostic experience from source diseases to target ones with a handful of data for adaptation. It is capitalized on a commonsense knowledge graph to characterize the …

Graph-Evolving Meta-Learning For Low-Resource Medical Dialogue Generation, Shuai Lin, Pan Zhou, Xiaodan Liang, Jianheng Tang, Ruihui Zhao, Ziliang Chen, Liang Lin

Research Collection School Of Computing and Information Systems

Human doctors with well-structured medical knowledge can diagnose a disease merely via a few conversations with patients about symptoms. In contrast, existing knowledge-grounded dialogue systems often require a large number of dialogue instances to learn as they fail to capture the correlations between different diseases and neglect the diagnostic experience shared among them. To address this issue, we propose a more natural and practical paradigm, i.e., low-resource medical dialogue generation, which can transfer the diagnostic experience from source diseases to target ones with a handful of data for adaptation. It is capitalized on a commonsense knowledge graph to characterize the …

Efficientderain: Learning Pixel-Wise Dilation Filtering For High-Efficiency Single-Image Deraining, Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu, Jianjun Zhao

Research Collection School Of Computing and Information Systems

Single-image deraining is rather challenging due to the unknown rain model. Existing methods often make specific assumptions of the rain model, which can hardly cover many diverse circumstances in the real world, compelling them to employ complex optimization or progressive refinement. This, however, significantly affects these methods’ efficiency and effectiveness for many efficiency-critical applications. To fill this gap, in this paper, we regard the single-image deraining as a general image-enhancing problem and originally propose a model-free deraining method, i.e., EfficientDeRain, which is able to process a rainy image within 10 ms (i.e., around 6 ms on average), over 80 times …

Three-Dimensional (3d) Reconstruction Of Computer Tomography Cardiac Images Using Visualization Toolkit (Vtk), Mohammad Zahid Zamaludin

Student Works (2020-2029)

Cardiac study proves to be important to diagnose and treat cardiovascular disease that accounts for 20% of death in the world. Three-dimensional (3D) images of the heart are important to understand its real structure, functions and physical abnormalities occurring in the tissue. The objective of this paper is to reconstruct 3D images from 2D CT Scan slices of the cardiac region of interest using Visualisation Toolkit (VTK). The Visualization Toolkit (VTK) is non-propriety software with state-of-the-art tools for 3D rendering that is easily customizable programming. 3D images are reconstructed using surface rendering technique and volume rendering technique. Volume rendering shows …

Evaluating Grasping Visualizations And Control Modes In A Vr Game, Alex Adkins, Lorraine Lin, Aline Normoyle, Ryan Canales, Yuting Ye, Sophie Jörg

Computer Science Faculty Research and Scholarship

A primary goal of the Virtual Reality(VR) community is to build fully immersive and presence-inducing environments with seamless and natural interactions. To reach this goal, researchers are investigating how to best directly use our hands to interact with a virtual environment using hand tracking. Most studies in this field require participants to perform repetitive tasks. In this article, we investigate if results of such studies translate into a real application and game-like experience. We designed a virtual escape room in which participants interact with various objects to gather clues and complete puzzles. In a between-subjects study, we examine the effects …

Human Activity Recognition Based On Wearable Flex Sensor And Pulse Sensor, Xiaozhu Jin

Electronic Theses and Dissertations

In order to fulfill the needs of everyday monitoring for healthcare and emergency advice, many HAR systems have been designed [1]. Based on the healthcare purpose, these systems can be implanted into an astronaut’s spacesuit to provide necessary life movement monitoring and healthcare suggestions. Most of these systems use acceleration data-based data record as human activity representation [2,3]. But this data attribute approach has a limitation that makes it impossible to be used as an activity monitoring system for astronavigation. Because an accelerometer senses acceleration by distinguishing acceleration data based on the earth’s gravity offset [4], the accelerometer cannot read …

R2u3d: Recurrent Residual 3d U-Net For Lung Segmentation, Dhaval D. Kadia, Md Zahangir Alom, Ranga Burada, Tam Nguyen, Vijayan K. Asari

Computer Science Faculty Publications

3D Lung segmentation is essential since it processes the volumetric information of the lungs, removes the unnecessary areas of the scan, and segments the actual area of the lungs in a 3D volume. Recently, the deep learning model, such as U-Net outperforms other network architectures for biomedical image segmentation. In this paper, we propose a novel model, namely, Recurrent Residual 3D U-Net (R(2)U3D), for the 3D lung segmentation task. In particular, the proposed model integrates 3D convolution into the Recurrent Residual Neural Network based on U-Net. It helps learn spatial dependencies in 3D and increases the propagation of 3D volumetric …

Smart Scribbles For Image Matting, Yang Xin, Yu Qiao, Shaozhe Chen, Shengfeng He, Baocai Yin, Qiang Zhang, Xiaopeng Wei, Rynson W. H. Lau

Research Collection School Of Computing and Information Systems

Image matting is an ill-posed problem that usually requires additional user input, such as trimaps or scribbles. Drawing a fine trimap requires a large amount of user effort, while using scribbles can hardly obtain satisfactory alpha mattes for non-professional users. Some recent deep learning-based matting networks rely on large-scale composite datasets for training to improve performance, resulting in the occasional appearance of obvious artifacts when processing natural images. In this article, we explore the intrinsic relationship between user input and alpha mattes and strike a balance between user effort and the quality of alpha mattes. In particular, we propose an …

Coherence And Identity Learning For Arbitrary-Length Face Video Generation, Shuquan Ye, Chu Han, Jiaying Lin, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Face synthesis is an interesting yet challenging task in computer vision. It is even much harder to generate a portrait video than a single image. In this paper, we propose a novel video generation framework for synthesizing arbitrary-length face videos without any face exemplar or landmark. To overcome the synthesis ambiguity of face video, we propose a divide-and-conquer strategy to separately address the video face synthesis problem from two aspects, face identity synthesis and rearrangement. To this end, we design a cascaded network which contains three components, Identity-aware GAN (IA-GAN), Face Coherence Network, and Interpolation Network. IA-GAN is proposed to …

Integrating The Bullet Physics Engine Into Minecraft, Ethan Johnson

Student Academic Conference

During the past fall semester, I started a programming project called Rayon which is designed to be a realistic physics engine implementation that runs alongside the videogame Minecraft. It is a library which Minecraft mod developers can use to implement realistic entity movement into their own mods. Rayon, being entirely written in the Java programming language, currently uses a port of the Bullet physics engine called JBullet which is very outdated and no longer being maintained. To find a more performant solution, I have set out to replace JBullet with an alternative library called LibBulletJME which is designed to interface …

Recapture: A Virtual Reality Interactive Narrative Experience Concerning Perspectives And Self-Reflection, Indira Avendano

Honors Undergraduate Theses

This project presents a virtual reality (VR) Interactive Narrative aiming to leave users reflecting on the perspectives one chooses to view life through. The narrative is driven by interactions designed using the concept of procedural rhetoric, which explores how rules and mechanics in games can persuade people about an idea, and Shin's cognitive model, which presents a dynamic view of immersion in VR. The persuasive nature of procedural rhetoric in combination with immersion techniques such as tangible interfaces and first-person elements of VR can effectively work together to immerse users into a compelling narrative experience with an intended emotional response …

Natural User Interface Based American Sign Language Tutoring Program, Bryce J. Allen

Williams Honors College, Honors Research Projects

The COVID-19 pandemic has exposed a substantial shortcoming in the modern American educational system: there is a sufficient need for our educators to be trained in the practices required to provide an educational experience for their students that is as effective as in-person instruction. There exist already systems of online instruction for various academic subjects, such as math and the sciences. In the subject of linguistic studies, educational programs have been developed to evaluate student proficiency in both the written and spoken forms of the language in which they are studying. However, there exist few programs that can effectively provide …

Interactive Virtual Reality Reading Experience, Nathaniel Shetler

Williams Honors College, Honors Research Projects

The project is an interactive virtual reality reading experience. The user is able to read a book or story in VR. When certain achievements are reached, such as finishing a chapter, the user is given the opportunity to transport to the environment that they are reading about. This gives the user a great opportunity to interact and learn hands-on with the material they are reading about. For example, if the user is reading about World War I, they will be given the opportunity to transport to the battlefields/trenches in Europe.

Infinite-Duration All-Pay Bidding Games, Guy Avni, Ismäel Jecker, Dorde Zikelic

Research Collection School Of Computing and Information Systems

In a two-player zero-sum graph game the players move a token throughout a graph to produce an infinite path, which determines the winner or payoff of the game. Traditionally, the players alternate turns in moving the token. In bidding games, however, the players have budgets, and in each turn, we hold an "auction" (bidding) to determine which player moves the token: both players simultaneously submit bids and the higher bidder moves the token. The bidding mechanisms differ in their payment schemes. Bidding games were largely studied with variants of first-price bidding in which only the higher bidder pays his bid. …

"Who Can Help Me?'': Knowledge Infused Matching Of Support Seekers And Support Providers During Covid-19 On Reddit, Manas Gaur, Kaushik Roy, Aditya Sharma, Biplav Srivastava, Amit Sheth

Publications

During the ongoing COVID-19 crisis, subreddits on Reddit, such as r/Coronavirus saw a rapid growth in user's requests for help (support seekers - SSs) including individuals with varying professions and experiences with diverse perspectives on care (support providers - SPs). Currently, knowledgeable human moderators match an SS with a user with relevant experience, i.e, an SP on these subreddits. This unscalable process defers timely care. We present a medical knowledge-infused approach to efficient matching of SS and SPs validated by experts for the users affected by anxiety and depression, in the context of with COVID-19. After matching, each SP to …

Virtual Tutor Personality In Computer Assisted Language Learning, Johanna Dobbriner, Cathy Ennis, Robert J. Ross

Conference papers

The use of intelligent virtual agents in language learning has increased in recent years. Studies into several aspects of personalisation aiming to increase user engagement are an ongoing research topic with avatar personality being one such aspect. As a step towards our development of intelligent virtual avatars, we present two of our initial experiments to explore differences in user interaction with two contrasting avatar personalities -- P1: open-minded, friendly and sociable and P2: closed-off, curt and distant. Each user interacted with a single personality in a video-call setting and gave feedback on the interaction. Our expectations, that P1 would be …

3d Dental Biometrics: Automatic Pose-Invariant Dental Arch Extraction And Matching, Xin Zhong, Zhiyuan Zhang

Research Collection School Of Computing and Information Systems

A novel automatic pose-invariant dental arch extraction and matching framework is developed for 3D dental identification using laser-scanned dental plasters. In our previous attempt [1-5], 3D point-based algorithms have been developed and they have shown a few advantages over existing 2D dental identifications. This study is a continuous effort in developing arch-based algorithms to extract and match dental arch feature in an automatic and pose-invariant way. As best as we know, this is the first attempt at automatic dental arch extraction and matching for 3D dental identification. A Radial Ray Algorithm (RRA) is proposed by projecting dental arch shape from …

Converting Optical Videos To Infrared Videos Using Attention Gan And Its Impact On Target Detection And Classification Performance, Mohammad Shahab Uddin, Reshad Hoque, Kazi Aminul Islam, Chiman Kwan, David Gribben, Jiang Li

Electrical & Computer Engineering Faculty Publications

To apply powerful deep-learning-based algorithms for object detection and classification in infrared videos, it is necessary to have more training data in order to build high-performance models. However, in many surveillance applications, one can have a lot more optical videos than infrared videos. This lack of IR video datasets can be mitigated if optical-to-infrared video conversion is possible. In this paper, we present a new approach for converting optical videos to infrared videos using deep learning. The basic idea is to focus on target areas using attention generative adversarial network (attention GAN), which will preserve the fidelity of target areas. …