Computer Sciences Commons^™

Learning From The Master: Distilling Cross-Modal Advanced Knowledge For Lip Reading, Sucheng Ren, Yong Du, Jianming Lv, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Lip reading aims to predict the spoken sentences from silent lip videos. Due to the fact that such a vision task usually performs worse than its counterpart speech recognition, one potential scheme is to distill knowledge from a teacher pretrained by audio signals. However, the latent domain gap between the cross-modal data could lead to a learning ambiguity and thus limits the performance of lip reading. In this paper, we propose a novel collaborative framework for lip reading, and two aspects of issues are considered: 1) the teacher should understand bi-modal knowledge to possibly bridge the inherent cross-modal gap; 2) …

Delving Deep Into Many-To-Many Attention For Few-Shot Video Object Segmentation, Haoxin Chen, Hanjie Wu, Nanxuan Zhao, Sucheng Ren, Shengfeng He

Research Collection School Of Computing and Information Systems

This paper tackles the task of Few-Shot Video Object Segmentation (FSVOS), i.e., segmenting objects in the query videos with certain class specified in a few labeled support images. The key is to model the relationship between the query videos and the support images for propagating the object information. This is a many-to-many problem and often relies on full-rank attention, which is computationally intensive. In this paper, we propose a novel Domain Agent Network (DAN), breaking down the full-rank attention into two smaller ones. We consider one single frame of the query video as the domain agent, bridging between the support …

Ganmut: Learning Interpretable Conditional Space For A Gamut Of Emotions, S. D'Apolito, D.P. Paundel, Zhiwu Huang, A.R. Vergara, Gool L. Van

Research Collection School Of Computing and Information Systems

Humans can communicate emotions through a plethora of facial expressions, each with its own intensity, nuances and ambiguities. The generation of such variety by means of conditional GANs is limited to the expressions encoded in the used label system. These limitations are caused either due to burdensome labeling demand or the confounded label space. On the other hand, learning from inexpensive and intuitive basic categorical emotion labels leads to limited emotion variability. In this paper, we propose a novel GAN-based framework which learns an expressive and interpretable conditional space (usable as a label space) of emotions, instead of conditioning on …

When Program Analysis Meets Bytecode Search: Targeted And Efficient Inter-Procedural Analysis Of Modern Android Apps In Backdroid, Daoyuan Wu, Debin Gao, Robert H. Deng, Rocky Chang

Research Collection School Of Computing and Information Systems

Widely-used Android static program analysis tools,e.g., Amandroid and FlowDroid, perform the whole-app interprocedural analysis that is comprehensive but fundamentallydifficult to handle modern (large) apps. The average app size hasincreased three to four times over five years. In this paper, weexplore a new paradigm of targeted inter-procedural analysis thatcan skip irrelevant code and focus only on the flows of securitysensitive sink APIs. To this end, we propose a technique calledon-the-fly bytecode search, which searches the disassembled appbytecode text just in time when a caller needs to be located. In thisway, it guides targeted (and backward) inter-procedural analysisstep by step until reaching …

Iquant: Interactive Quantitative Investment Using Sparse Regression Factors, Xuanwu Yue, Qiao Gu, Deyun Wang, Huamin Qu, Yong Wang

Research Collection School Of Computing and Information Systems

The model-based investing using financial factors is evolving as a principal method for quantitative investment. The main challenge lies in the selection of effective factors towards excess market returns. Existing approaches, either hand-picking factors or applying feature selection algorithms, do not orchestrate both human knowledge and computational power. This paper presents iQUANT, an interactive quantitative investment system that assists equity traders to quickly spot promising financial factors from initial recommendations suggested by algorithmic models, and conduct a joint refinement of factors and stocks for investment portfolio composition. We work closely with professional traders to assemble empirical characteristics of “good” factors …

Prediction Of Sleep Quality In Live-Alone Diabetic Seniors Using Unobtrusive In-Home Sensors, Barry Nuqoba, Hwee-Pink Tan

Research Collection School Of Computing and Information Systems

Diabetes, a chronic disease that occurs when the pancreas does not produce enough insulin or when the body cannot effectively utilize its insulin, is increasingly recognized as a significant health burden and affects many older adults. Poor sleep quality in diabetic seniors worsens the diabetes condition, but most seniors are tend to regard poor sleep quality as a usual event and do not seek treatment. This study aims to detect poor sleep quality in diabetic seniors through passive in-home monitoring to inform intervention (e.g., seeking diagnosis and treatment) to improve the physical and mental health of diabetic seniors. We derive …

Applying Facial Emotion Recognition To Usability Evaluations To Reduce Analysis Time, Gavin Kam Chao

Master's Theses

Usability testing is an important part of product design that offers developers insight into a product’s ability to help users achieve their goals. Despite the usefulness of usability testing, human usability evaluations are costly and time-intensive processes. Developing methods to reduce the time and costs of usability evaluations is important for organizations to improve the usability of their products without expensive investments. One prospective solution to this is the application of facial emotion recognition to automate the collection of qualitative metrics normally identified by human usability evaluators.

In this paper, facial emotion recognition (FER) was applied to mock usability recordings …

An Analysis Of Real-Time Ray Tracing Techniques Using The Vulkan® Explicit Api, Elleis C. Souza

Master's Theses

In computer graphics applications, the choice and implementation of a rendering technique is crucial when targeting real-time performance. Traditionally, rasterization-based approaches have dominated the real-time sector. Other algorithms were simply too slow to compete on consumer graphics hardware. With the addition of hardware support for ray-intersection calculations on modern GPUs, hybrid ray tracing/rasterization and purely ray tracing approaches have become possible in real-time as well. Industry real-time graphics applications, namely games, have been exploring these different rendering techniques with great levels of success. The addition of ray tracing into the graphics developer’s toolkit has without a doubt increased what level …

A Performance Survey Of Text-Based Sentiment Analysis Methods For Automating Usability Evaluations, Kelsi Van Damme

Master's Theses

Usability testing, or user experience (UX) testing, is increasingly recognized as an important part of the user interface design process. However, evaluating usability tests can be expensive in terms of time and resources and can lack consistency between human evaluators. This makes automation an appealing expansion or alternative to conventional usability techniques.

Early usability automation focused on evaluating human behavior through quantitative metrics but the explosion of opinion mining and sentiment analysis applications in recent decades has led to exciting new possibilities for usability evaluation methods.

This paper presents a survey of modern, open-source sentiment analyzers’ usefulness in extracting and …

Visualizing Decision Trees And Forests Using Radial Trees, Angela Kozma

Computer Science and Information Technology Faculty

Data visualization has become a big representation of many company’s data and schedules. Now people are not using just simple bar graphs and pie charts in business meetings but utilizing other fields of study and even more complex graphs. By using multiple visualizations to display their results and projects, it is letting more outside people understand what they are working on and can lead to more viewpoints on the topic being displayed. Also, schedules for projects are now being displayed visually so the workers can see how much time each part of their project is going to take. With this …

A Hybrid Gaze Pointer With Voice Control, Indhuja Ravi

Master's Projects

Accessibility in technology has been a challenge since the beginning of the 1800s. Starting with building typewriters for the blind by Pellegrino Turri to the on-screen keyboard built by Microsoft, there have been several advancements towards assistive technologies. The basic tools necessary for anyone to operate a computer are to be able to navigate the device, input information, and perceive the output. All these three categories have been undergoing tremendous advancements over the years. Especially, with the internet boom, it has now become a necessity to point onto a computer screen. This has somewhat attracted research into this particular area. …

Musical Gesture Through The Human Computer Interface: An Investigation Using Information Theory, Michael Vincent Blandino

LSU Doctoral Dissertations

This study applies information theory to investigate human ability to communicate using continuous control sensors with a particular focus on informing the design of digital musical instruments. There is an active practice of building and evaluating such instruments, for instance, in the New Interfaces for Musical Expression (NIME) conference community. The fidelity of the instruments can depend on the included sensors, and although much anecdotal evidence and craft experience informs the use of these sensors, relatively little is known about the ability of humans to control them accurately. This dissertation addresses this issue and related concerns, including continuous control performance …

The Social Market Economy As A Formula For Peace, Prosperity, And Sustainability, Almuth D. Merkel

Doctor of International Conflict Management Dissertations

The social market economy was developed in Germany during the interwar period amidst political and economic turmoil. With clear demarcation lines differentiating it from socialism and laissez-faire capitalism, the social market economy became a formula for peace and prosperity for post WWII Germany. Since then, the success of the social market economy has inspired many other countries to adopt its principles. Drawing on evidence from economic history and the history of economic thought, this thesis first reviews the evolution of the fundamental principles that form the foundation of social-market economic thought. Blending the micro-economic utility maximization framework with traditional growth …

Analysis Of Students’ Multi-Representation Ability In Augmented Reality-Assisted Learning, Sri Jumini, Edy Cahyono, Muhamad Miftakhul Falah

Library Philosophy and Practice (e-journal)

Not all learning sources can directly and cheaply be presented, so augmented reality media is needed to be applied to students with various talents and intelligence. This study aims to analyze students’ multi-representation ability through the use of augmented reality media. The research method was carried out through pre-experiment with one group posttest only design. Test question items were given to see the students’ multi-representation ability. Data analysis was carried out through the percentage of the number of students achieving test scores of more than or equal to 80 on a scale of 100. The results showed that 88% (28 …

Learning Intermediate Representations For Question Answering Systems, Zakery T. Clarke

Computer Science ETDs

Question answering systems are models that can perform natural language processing (NLP) on a question, retrieve an answer from a datasource, and communicate it to a user. In question answering systems, it is important for the system to learn an underlying representation for a piece of text. There are many systems that have achieved incredible accuracy on question answering datasets such as the Stanford Question and Answer Dataset (SQuAD), but these systems often encode their knowledge in a manner that is impossible to verify. Many current models would benefit more from verifiability, than marginal accuracy improvements.

We propose a method …

City Goers: An Exploration Into Creating Seemingly Intelligent A.I. Systems, Matthew Brooke

Computer Science and Computer Engineering Undergraduate Honors Theses

Artificial Intelligence systems have come a long way over the years. One particular application of A.I. is its incorporation in video games. A key goal of creating an A.I. system in a video game is to convey a level of intellect to the player. During playtests for Halo: Combat Evolved, the developers at Bungie noticed that players deemed tougher enemies as more intelligent than weaker ones, despite the fact that there were no differences in behavior in the enemies. The tougher enemies provided a greater illusion of intelligence to the players. Inspired by this, I set out to create a …

Security Fatigue And Its Effects On Perceived Password Strength Among University Students, Chase Carroll

Honors Theses

This study was performed with the goal of observing the effect, if any, that security fatigue has on students’ perceived strength of passwords. In doing so, it was hoped to find some correlation between the two that would help in establishing a measurable effect of the phenomenon in students. This could potentially aid organizational decision-makers, such as security policy writers and system admins, to make more informed decisions about implementing security measures. To achieve the goal of observing this fatigue and attempting to measure it, a survey was distributed to numerous students on the University of Tennessee at Chattanooga campus. …

Brave New World Reboot: Technology’S Role In Consumer Manipulation And Implications For Privacy And Transparency, Allie Mertensotto

Marketing Undergraduate Honors Theses

Most consumers are aware that our data is being obtained and collected through the use of our devices we keep in our homes or even on our person throughout the day. But, it is understated how much data is being collected. Conversations you have with your peers – in a close proximity of a device – are being used to tailor advertising. The advertisements you receive on your devices are uniquely catered to your individual person, due to the fact it consistently uses our data to produce efficient and personal ads. On the flip side, our government is also tapping …

The Effect Of User Interface, User Experience And Design On Mobile E-Commerce Applications In The Fashion Industry, Debra Perlman

Honors College Theses

This study aims to explore elements of the user experience, and how those with the user interface and design of mobile apps in the e-commerce and fashion industries affect the user experience and customer behavior with online shopping. In this area, other work has been done in evaluating the user experience, and its effects on components of the user interface and design, however, this study offers a more specific look into these features in a niche industry, namely in the frame of e-commerce and the fashion industry. This fits into the context of the field of the user experience as …

Achieving Differential Privacy And Fairness In Machine Learning, Depeng Xu

Graduate Theses and Dissertations

Machine learning algorithms are used to make decisions in various applications, such as recruiting, lending and policing. These algorithms rely on large amounts of sensitive individual information to work properly. Hence, there are sociological concerns about machine learning algorithms on matters like privacy and fairness. Currently, many studies only focus on protecting individual privacy or ensuring fairness of algorithms separately without taking consideration of their connection. However, there are new challenges arising in privacy preserving and fairness-aware machine learning. On one hand, there is fairness within the private model, i.e., how to meet both privacy and fairness requirements simultaneously in …

Prediction, Recommendation And Group Analytics Models In The Domain Of Mashup Services And Cyber-Argumentation Platform, Md Mahfuzer Rahman

Graduate Theses and Dissertations

Mashup application development is becoming a widespread software development practice due to its appeal for a shorter application development period. Application developers usually use web APIs from different sources to create a new streamlined service and provide various features to end-users. This kind of practice saves time, ensures reliability, accuracy, and security in the developed applications. Mashup application developers integrate these available APIs into their applications. Still, they have to go through thousands of available web APIs and chose only a few appropriate ones for their application. Recommending relevant web APIs might help application developers in this situation. However, very …

Effect Of Augmented Reality On Consumer Behavior In E-Commerce, Chibuke Uzoechina, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

AR technology has been referred to as the future of e-commerce. In this paper, we propose to investigate the effect of AR on consumer behavior in e-commerce. Based on the theory of telepresence, we generated a set of hypotheses on their relationships. An experimental study is proposed to test the hypotheses. Our findings will be of interest to e-commerce companies that are looking at incorporating AR into their e-commerce platforms.

Prototypical Contrastive Learning Of Unsupervised Representations, Junnan Li, Pan Zhou, Caiming Xiong, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

This paper presents Prototypical Contrastive Learning (PCL), an unsupervised representation learning method that bridges contrastive learning with clustering. PCL not only learns low-level features for the task of instance discrimination, but more importantly, it encodes semantic structures discovered by clustering into the learned embedding space. Specifically, we introduce prototypes as latent variables to help find the maximum-likelihood estimation of the network parameters in an Expectation-Maximization framework. We iteratively perform E-step as finding the distribution of prototypes via clustering and M-step as optimizing the network via contrastive learning. We propose ProtoNCE loss, a generalized version of the InfoNCE loss for contrastive …

Cross-Modal Food Retrieval: Learning A Joint Embedding Of Food Images And Recipes With Semantic Consistency And Attention Mechanism;, Hao Wang, Doyen Sahoo, Chenghao Liu, Ke Shu, Achananuparp Palakorn, Ee Peng Lim, Steven Hoi

Research Collection School Of Computing and Information Systems

Food retrieval is an important task to perform analysis of food-related information, where we are interested in retrieving relevant information about the queried food item such as ingredients, cooking instructions, etc. In this paper, we investigate cross-modal retrieval between food images and cooking recipes. The goal is to learn an embedding of images and recipes in a common feature space, such that the corresponding image-recipe embeddings lie close to one another. Two major challenges in addressing this problem are 1) large intra-variance and small inter-variance across cross-modal food data; and 2) difficulties in obtaining discriminative recipe representations. To address these …

More Kawaii Than A Real-Person Live Streamer: Understanding How The Otaku Community Engages With And Perceives Virtual Youtubers, Zhicong Lu, Chenxinran Shen, Jiannan Li, Hong Shen, Daniel Wigdor

Research Collection School Of Computing and Information Systems

Live streaming has become increasingly popular, with most streamers presenting their real-life appearance. However, Virtual YouTubers (VTubers), virtual 2D or 3D avatars that are voiced by humans, are emerging as live streamers and attracting a growing viewership in East Asia. Although prior research has found that many viewers seek real-life interpersonal interactions with real-person streamers, it is currently unknown what makes VTuber live streams engaging or how they are perceived differently than real-person streamers. We conducted an interview study to understand how viewers engage with VTubers and perceive the identities of the voice actors behind the avatars (i.e., Nakanohito). The …

Edgeduet: Tiling Small Object Detection For Edge Assisted Autonomous Mobile Vision, Xu Wang, Zheng Yang, Jiahang Wu, Yi Zhao, Zimu Zhou

Research Collection School Of Computing and Information Systems

Accurate, real-time object detection on resource-constrained devices enables autonomous mobile vision applications such as traffic surveillance, situational awareness, and safety inspection, where it is crucial to detect both small and large objects in crowded scenes. Prior studies either perform object detection locally on-board or offload the task to the edge/cloud. Local object detection yields low accuracy on small objects since it operates on low-resolution videos to fit in mobile memory. Offloaded object detection incurs high latency due to uploading high-resolution videos to the edge/cloud. Rather than either pure local processing or offloading, we propose to detect large objects locally while …

Analog Spiking Neural Network Implementing Spike Timing-Dependent Plasticity On 65 Nm Cmos, Luke Vincent

Graduate Theses and Dissertations

Machine learning is a rapidly accelerating tool and technology used for countless applications in the modern world. There are many digital algorithms to deploy a machine learning program, but the most advanced and well-known algorithm is the artificial neural network (ANN). While ANNs demonstrate impressive reinforcement learning behaviors, they require large power consumption to operate. Therefore, an analog spiking neural network (SNN) implementing spike timing-dependent plasticity is proposed, developed, and tested to demonstrate equivalent learning abilities with fractional power consumption compared to its digital adversary.

Visuo-Haptic Illusions For Linear Translation And Stretching Using Physical Proxies In Virtual Reality, Martin Feick, Niko Kleer, André Zenner, Anthony Tang, Antonio Kruger

Research Collection School Of Computing and Information Systems

Providing haptic feedback when manipulating virtual objects is an essential part of immersive virtual reality experiences; however, it is challenging to replicate all of an object’s properties and characteristics. We propose the use of visuo-haptic illusions alongside physical proxies to enhance the scope of proxy-based interactions with virtual objects. In this work, we focus on two manipulation techniques, linear translation and stretching across different distances, and investigate how much discrepancy between the physical proxy and the virtual object may be introduced without participants noticing. In a study with 24 participants, we found that manipulation technique and travel distance significantly affect …

A Visual Analytics Approach To Facilitate The Proctoring Of Online Exams, Haotian Li, Min Xu, Yong Wang, Huan Wei, Huamin Qu

Research Collection School Of Computing and Information Systems

Online exams have become widely used in recent years to evaluate students’ performance in mastering the knowledge, especially during the pandemic of COVID-19. However, it is challenging to conduct proctoring for online exams due to the lack of face-to-face interactions. Also, prior research has shown that online exams are more vulnerable to various cheating behaviors, which can damage the credibility of online exams. In this paper, we present a novel vi- sual analytics approach to facilitate the proctoring of online exams by analyzing the exam video records and mouse movement data of each student. Specifically, we detect and visualize suspected …

Interactive Search Vs. Automatic Search: An Extensive Study On Video Retrieval, Phuong-Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

This article conducts user evaluation to study the performance difference between interactive and automatic search. Particularly, the study aims to provide empirical insights of how the performance landscape of video search changes, with tens of thousands of concept detectors freely available to exploit for query formulation. We compare three types of search modes: free-to-play (i.e., search from scratch), non-free-to-play (i.e., search by inspecting results provided by automatic search), and automatic search including concept-free and concept-based retrieval paradigms. The study involves a total of 40 participants; each performs interactive search over 15 queries of various difficulty levels using two search modes …