Digital Commons Network^™

Friendly Sharpness-Aware Minimization, Tao Li, Pan Zhou, Zhengbao He, Xinwen Cheng, Xiaolin Huang

Research Collection School Of Computing and Information Systems

Sharpness-Aware Minimization (SAM) has been instrumental in improving deep neural network training by minimizing both training loss and loss sharpness. Despite the practical success, the mechanisms behind SAM’s generalization enhancements remain elusive, limiting its progress in deep learning optimization. In this work, we investigate SAM’s core components for generalization improvement and introduce “Friendly-SAM” (F-SAM) to further enhance SAM’s generalization. Our investigation reveals the key role of batch-specific stochastic gradient noise within the adversarial perturbation, i.e., the current minibatch gradient, which significantly influences SAM’s generalization performance. By decomposing the adversarial perturbation in SAM into full gradient and stochastic gradient noise components, …

Few-Shot Learner Parameterization By Diffusion Time-Steps, Zhongqi Yue, Pan Zhou, Richang Hong, Hanwang Zhang, Sun Qianru

Research Collection School Of Computing and Information Systems

Even when using large multi-modal foundation models, few-shot learning is still challenging—if there is no proper inductive bias, it is nearly impossible to keep the nuanced class attributes while removing the visually prominent attributes that spuriously correlate with class labels. To this end, we find an inductive bias that the time-steps of a Diffusion Model (DM) can isolate the nuanced class attributes, i.e., as the forward diffusion adds noise to an image at each time-step, nuanced attributes are usually lost at an earlier time-step than the spurious attributes that are visually prominent. Building on this, we propose Time-step Few-shot (TiF) …

Diffusion Time-Step Curriculum For One Image To 3d Generation, Xuanyu Yi, Zike Wu, Qingshan Xu, Pan Zhou, Joo Hwee Lim, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Score distillation sampling (SDS) has been widely adopted to overcome the absence of unseen views in reconstructing 3D objects from a single image. It leverages pretrained 2D diffusion models as teacher to guide the reconstruction of student 3D models. Despite their remarkable success, SDS-based methods often encounter geometric artifacts and texture saturation. We find out the crux is the overlooked indiscriminate treatment of diffusion time-steps during optimization: it unreasonably treats the studentteacher knowledge distillation to be equal at all time-steps and thus entangles coarse-grained and fine-grained modeling. Therefore, we propose the Diffusion Time-step Curriculum one-image-to-3D pipeline (DTC123), which involves both …

Enhancing Code Vulnerability Detection Via Vulnerability-Preserving Data Augmentation, Shangqing Liu, Wei Ma, Jian Wang, Xiaofei Xie, Ruitao Feng, Yang Liu

Research Collection School Of Computing and Information Systems

Source code vulnerability detection aims to identify inherent vulnerabilities to safeguard software systems from potential attacks. Many prior studies overlook diverse vulnerability characteristics, simplifying the problem into a binary (0-1) classification task for example determining whether it is vulnerable or not. This poses a challenge for a single deep-learning based model to effectively learn the wide array of vulnerability characteristics. Furthermore, due to the challenges associated with collecting large-scale vulnerability data, these detectors often overfit limited training datasets, resulting in lower model generalization performance. To address the aforementioned challenges, in this work, we introduce a fine-grained vulnerability detector namely FGVulDet. …

Detecting Foot Strikes During Running With Earbuds, Changshuo Hu, Thivya Kandappu, Jake Stuchbury-Wass, Yang Liu, Anthony Tang, Cecelia Mascolo, Dong Ma

Research Collection School Of Computing and Information Systems

Running is a widely embraced form of aerobic exercise, offering various physical and mental benefits. However, improper running gaits (i.e., the way of foot landing) can pose safety risks and impact running efficiency. As many runners lack the knowledge or continuous attention to manage their foot strikes during running, in this work, we present a portable and non-invasive running gait monitoring system. Specifically, we leverage the in-ear microphone on wireless earbuds to capture the vibrations generated by foot strikes. Landing with different parts of the foot (e.g., forefoot and heel) generates distinct vibration patterns, and thus we utilize machine learning …

Gts: Gpu-Based Tree Index For Fast Similarity Search, Yifan Zhu, Ruiyao Ma, Baihua Zheng, Xiangyu Ke, Lu Chen, Yunjun Gao

Research Collection School Of Computing and Information Systems

Similarity search, the task of identifying objects most similar to a given query object under a specific metric, has gathered significant attention due to its practical applications. However, the absence of coordinate information to accelerate similarity search and the high computational cost of measuring object similarity hinder the efficiency of existing CPU-based methods. Additionally, these methods struggle to meet the demand for high throughput data management. To address these challenges, we propose GTS, a GPU-based tree index designed for the parallel processing of similarity search in general metric spaces, where only the distance metric for measuring object similarity is known. …

Jollygesture: Exploring Dual-Purpose Gestures In Vr Presentations, Gun Woo Warren Park, Anthony Tang, Fanny Chevalier

Research Collection School Of Computing and Information Systems

Virtual reality (VR) offers new opportunities for presenters to use expressive body language to engage their audience. Yet, most VR presentation systems have adopted control mechanisms that mimic those found in face-to-face presentation systems. We explore the use of gestures that have dual-purpose: first, for the audience, a communicative purpose; second, for the presenter, a control purpose to alter content in slides. To support presenters, we provide guidance on what gestures are available and their effects. We realize our design approach in JollyGesture, a VR technology probe that recognizes dual-purpose gestures in a presentation scenario. We evaluate our approach through …

How Is Our Mobility Affected As We Age? Findings From A 934 Users Field Study Of Older Adults Conducted In An Urban Asian City, Yi Zhen Tan, Ngoc Doan Thu Tran, Sapphire Lin, Fang Zhao, Yee Sien Ng, Dong Ma, Jeonggil Ko, Rajesh Krishna Balan

Research Collection School Of Computing and Information Systems

In this paper, we analyze the results of a large study involving 934 older adults living in an urban Asian city that collected their mobility patterns, in the form of logged GPS data, along with a multitude of demographic and health data. We show that mobility, in terms of average distance travelled per day, is greatly affected by age and by employment status. In addition, other factors such as type of day, household size, physical and financial conditions and the onset of retirement also play a significant role in determining the mobility of an individual. These results will have high …

Usability Versus Collectibility In Nft: The Case Of Web3 Domain Names, Ping Fan Ke, Yi Meng Lau

Research Collection School Of Computing and Information Systems

This study examines the market’s inclination towards usability and collectibility aspects of Non-Fungible Tokens (NFTs) within Web3 domain name marketplaces, drawing insights from resale records. Our findings reveal a prevailing preference for usability, as evidenced by consistently higher average resale prices observed for Ethereum Name Service (ENS) domains compared to Linagee Name Registrar (LNR) domains. However, domains with diminished usability, such as those containing non-ASCII characters, tend to attract investors due to their enhanced collectibility. Our analysis on the effect from previous resale suggests a potential aversion towards second-hand acquisitions among NFT investors when value derives primarily from usability, while …

Poster: Profiling Event Vision Processing On Edge Devices, Ila Nitin Gokarn, Archan Misra

Research Collection School Of Computing and Information Systems

As RGB camera resolutions and frame-rates improve, their increased energy requirements make it challenging to deploy fast, efficient, and low-power applications on edge devices. Newer classes of sensors, such as the biologically inspired neuromorphic event-based camera, capture only changes in light intensity per-pixel to achieve operational superiority in sensing latency (O(μs)), energy consumption (O(mW)), high dynamic range (140dB), and task accuracy such as in object tracking, over traditional RGB camera streams. However, highly dynamic scenes can yield an event rate of up to 12MEvents/second, the processing of which could overwhelm …

Predicting Mild Cognitive Impairment Through Ambient Sensing And Artificial Intelligence, Ah-Hwee Tan, Weng Yan Ying, Budhitama Subagdja, Anni Huang, Shanthoshigaa D, Tony Chin-Ian Tay, Iris Rawtaer

Research Collection School Of Computing and Information Systems

This paper reports an emerging application leveraging ambient and artificial intelligence techniques for in-home sensing and cognitive health assessment. The application involves a prospective longitudinal study, wherein non-pervasive sensing devices are installed in homes of over 63 real users undergoing clinical cognitive assessment, and digital signals of the users’ activities and behaviour are transmitted to a central cloud-based data server for further processing and analysis. Based on the sensor readings, we identify a set of digital biomarkers covering four key aspects of daily living, namely physical, activity, cognitive, and sleep, and develop a suite of customized feature extraction methods for …

Sensing With Integrity: Responsible Sensor Systems In An Era Of Ai, David Eisenberg

Dissertations

Deep and machine learning now offer immense benefits for consumer choice, decision-making, medicine, mental health and education, smart cities, and intelligent transportation and driver safety. However, as communication and Internet technology further advances, these benefits have the potential to be outweighed by compromises to privacy, personal freedom, consumer trust, and discrimination. While ethical consequences for personal freedom and equity rise from these technological advances, the issue may not be the technology itself but a lack of regulation and policy that allow abuses to occur. A first study examines how emerging sensor-based technologies, limited to only accelerometer and gyroscope data from …

Charting A Path To The Quintuple Aim: Harnessing Ai To Address Social Determinants Of Health, Yash Shah, Zachary Goldberg, Erika Harness, David Nash

College of Population Health Faculty Papers

The Quintuple Aim seeks to improve healthcare by addressing social determinants of health (SDOHs), which are responsible for 70-80% of medical outcomes. SDOH-related concerns have traditionally been addressed through referrals to social workers and community-based organizations (CBOs), but these pathways have had limited success in connecting patients with resources. Given that health inequity is expected to cost the United States nearly USD 300 billion by 2050, new artificial intelligence (AI) technology may aid providers in addressing SDOH. In this commentary, we present our experience with using ChatGPT to obtain SDOH management recommendations for archetypal patients in Philadelphia, PA. ChatGPT identified …

Try It Together: Transcribing Your Audio With Whisper Api, Bella Ratmelia

2024 AI for Research Week

In this hands-on session, we will explore using the Whisper API to transcribe audio recordings from interviews, focus groups, and speeches. The session will delve into best practices and address common issues that may arise during the transcription process.

Intelligent Solutions For Retroactive Anomaly Detection And Resolution With Log File Systems, Derek G. Rogers, Chanvo Nguyen, Abhay Sharma

SMU Data Science Review

This paper explores the intricate challenges log files pose from data science and machine learning perspectives. Drawing inspiration from existing methods, LAnoBERT, PULL, LLMs, and the breadth of recent research, this paper aims to push the boundaries of machine learning for log file systems. Our study comprehensively examines the unique challenges presented in our problem setup, delineates the limitations of existing methods, and introduces innovative solutions. These contributions are organized to offer valuable insights, predictions, and actionable recommendations tailored for Microsoft's engineers working on log data analysis.

Leveraging Transformer Models For Genre Classification, Andreea C. Craus, Ben Berger, Yves Hughes, Hayley Horn

SMU Data Science Review

As the digital music landscape continues to expand, the need for effective methods to understand and contextualize the diverse genres of lyrical content becomes increasingly critical. This research focuses on the application of transformer models in the domain of music analysis, specifically in the task of lyric genre classification. By leveraging the advanced capabilities of transformer architectures, this project aims to capture intricate linguistic nuances within song lyrics, thereby enhancing the accuracy and efficiency of genre classification. The relevance of this project lies in its potential to contribute to the development of automated systems for music recommendation and genre-based playlist …

Automatic Measurement Of Dialogue Engagingness In Multilingual Settings, Amila Ferron

Dissertations and Theses

Expansive use of large language models (LLMs) as dialogue systems brings increased importance to the evaluation of the responses they generate. Although evaluation of qualities such as coherence and fluency are readily possible with well-established automatic metrics, engagingness is often measured with human evaluation -- a process that can be costly and slows the pace of development. Existing automatic metrics for engagingness have low to moderate correlation with human annotations, evaluate the response without the conversation history, are complicated to implement, or are designed for a specific dataset. Moreover, they have been tested exclusively on English conversations. Given that dialogue …

Academic Search And Discovery Tools In The Age Of Ai And Large Language Models: An Overview Of The Space, Aaron Tay

2024 AI for Research Week

In the ever-evolving landscape of academic research, “AI tools” for literature search and synthesis are currently getting a lot of attention. These tools promise to ramp up productivity, enabling us to accomplish more in less time or absorb more knowledge without drowning in endless reading. With the sheer number of these systems increasing daily, it's natural to wonder: are they really worth our time and money? And if they are, how should we go about picking the right one from the multitude of options?

In this talk, I will share my views on how the space has developed over two …

Crisis Management: Unveiling Information And Communication Technologies’ Revamped Role Through The Lens Of Sub-Saharan African Countries During Covid-19, Ruthbetha Kateule, Egidius Kamanyi, Mahadia Tunga

The African Journal of Information Systems

The management of COVID-19 pandemic has revealed inefficiencies in coordinating global response, particularly in African countries. Therefore, creating an urgent need to examine the literature on Information and Communication Technologies (ICT) in crisis management to appreciate its contextual role. Employing a systematic review, using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), this paper critically assessed the extent of the use of ICT in crisis management in Africa’s response to COVID-19 to reconstruct its resilience against future crises. Findings indicate that while countries with limited ICT infrastructure faced considerable challenges in utilizing ICT solutions in COVID-19 management, countries …

Advancing Objective Mobile Device Use Measurement Inchildren Ages 6–11 Through Built-In Device Sensors: A Proof-Of-Concept Study, Olivia L. Finnegan, Robert Glenn Weaver Med, Phd, Hongpeng Yang, James W. White, Srihari Nelakuditi, Zifei Zhong, Rahul Ghosal Ph.D., Yan Tong, Aliye B. Cepni, Elizabeth L. Adams, Sarah Burkart Mph, Ph.D., Michael W. Beets Med, Mph, Phd, Bridget Armstrong Ph.D.

Faculty Publications

Mobile devices (e.g., tablets and smartphones) have been rapidly integrated into the lives of children and have impacted howchildren engage with digital media. The portability of these devices allows for sporadic, on-demand interaction, reducing theaccuracy of self-report estimates of mobile device use. Passive sensing applications objectively monitor time spent on a givendevice but are unable to identify who is using the device, a significant limitation in child screen time research. Behavioralbiometric authentication, using embedded mobile device sensors to continuously authenticate users, could be applied toaddress this limitation. This study examined the preliminary accuracy of machine learning models trained on iPad …

Examining Differences In Concept Representation Across Similarity Spaces Between Humans And Large Language Models, Krishnachandra Nair

Computer Science Senior Theses

The replication of human concept representation is a critical task for the pursuit of artificial general intelligence. With the recent influx of large language models that demonstrate text-generation capabilities nearly on par with humans, the question stands on whether these large language models can capture concepts within language. We examine this question by exploring differences in concept representation across similarity spaces between humans and LLMs. We find that, while concept representation within LLMs does partially mimic human concept representation, LLMs are greatly limited by their dependence on semantic information and cannot therefore develop an understanding of human social code or …

Queering Futures With Data-Driven Speculation: The Design Of An Expanded Mixed Methods Research Framework Integrating Quantitative, Qualitative, And Practice-Based Modes, Jess Parris Westbrook

College of Education Theses and Dissertations

Queering’ questions, unlearns, disrupts, and transforms approaches, expectations, and realities. Futures are time and change. The approach I have designed to operationalize Queering and futures, or Queering futures, is the Queering Futures Framework (QFF). The Queering Futures Framework (QFF) is a brand-new transdisciplinary research framework intersecting values, positionality, complexity, Queerness themes, and futures praxis. This framework expands traditional mixed methods research conventions by integrating quantitative, qualitative, and practice-based research modes and mindsets. The Queering Futures Framework (QFF) prototype presented in this dissertation functions as a test case and proof of concept. The prototype quantitative mode measures attitudes towards AI, and …

A Deep Learning Framework For Blockage Mitigation In Mmwave Wireless, Ahmed Hazaa Almutairi

Dissertations and Theses

Millimeter-Wave (mmWave) communication is a key technology to enable next generation wireless systems. However, mmWave systems are highly susceptible to blockages, which can lead to a substantial decrease in signal strength at the receiver. Identifying blockages and mitigating them is thus a key challenge to achieve next generation wireless technology goals, such as enhanced mobile broadband (eMBB) and Ultra-Reliable and Low-Latency Communication (URLLC). This thesis proposes several deep learning (DL) frameworks for mmWave wireless blockage detection, mitigation, and duration prediction. First, we propose a DL framework to address the problem of identifying whether the mmWave wireless channel between two devices …

Text-To-Sql: A Methodical Review Of Challenges And Models, Ali Buğra Kanburoğlu, Faik Boray Tek

Turkish Journal of Electrical Engineering and Computer Sciences

This survey focuses on Text-to-SQL, automated translation of natural language queries into SQL queries. Initially, we describe the problem and its main challenges. Then, by following the PRISMA systematic review methodology, we survey the existing Text-to-SQL review papers in the literature. We apply the same method to extract proposed Text-to-SQL models and classify them with respect to used evaluation metrics and benchmarks. We highlight the accuracies achieved by various models on Text-to-SQL datasets and discuss execution-guided evaluation strategies. We present insights into model training times and implementations of different models. We also explore the availability of Text-to-SQL datasets in non-English …

Supporting South Korea’S Aging Population: How Ai And Iot Acceptance Connects The Young And Old, Bobby Im

Master's Projects and Capstones

In 2024, South Korea surpassed every other nation by becoming the country with the lowest fertility rate (below 0.7%). Population decline will hinder future ability to care for their aging population and although the government and private corporations are investing millions of dollars on developing Artificial Intelligence-Internet of Things (AI-IoT) devices to support the aging, the acceptance levels and the amount of family support required is undervalued. By examining AI-IoT’s current use and role in South Korea’s public health system this paper shows how intergenerational support helps optimize existing procedures and equipment, increases the level of acceptance and use, and …

Re-Instatement Of Pell Grants For Incarcerated Students: Implications For Cs Education, E. Hogan, D. J. Mir, A. Cencini, Keith O'Hara, A. G. S. Raj, W. G. Griswold, L. Porter

Computer Science Faculty Works

The US incarcerates more people, and at a higher rate, than any other country in the world—predominantly low-income people of color. Higher education in prison (HEP) has a powerful impact on justice-impacted people, improving their quality of life during incarceration and reducing their likelihood of returning to prison post-release. HEP nearly disappeared in 1994 after Pell Grant eligibility was eliminated for incarcerated students, which was recently reinstated. More computing educators can reach incarcerated students with digital literacy skills, core computing content, and critical analyses of the impacts of computing on society. CS education researchers can also support the growth of …

Prevention Of Web Scraping And Copy And Paste Of Content By Font Obfuscation, Mingkui Wei, Yao Liu, Zhuo Lu, Junjie Xiong

Computer Science Faculty Research & Creative Works

A method and system provide and utilize obfuscated fonts for displayable content. Responsive to a request for displayable content having text, a text portion of the requested displayable content to be obfuscated is determined. For that text portion, obfuscated fonts are provided, by retrieving obfuscated fonts or by generating obfuscated fonts from a set of obfuscated glyphs created from a plurality of glyphs representative of a plurality of characters of the text portion of the displayable content. The obfuscated fonts can be created by assigning obfuscated glyphs of the set into the obfuscated fonts in accordance with one or more …

Towards Machine Proficiency With Semantic Underspecification, Zachary S. Gottesman

Dartmouth College Master’s Theses

Human natural language communication frequently relies on extra-linguistic information to fill in gaps in the linguistic signal left by semantic underspecification, or the omission of details that can be inferred from prior knowledge or other modalities. Underspecification is particularly common in conversations between acquaintances, since these interlocutors share context. Underspecification is a key and beneficial feature of natural language that improves efficiency, although it can cause communication to fail if it is not resolved correctly. For language models to communicate effectively and in a human-like fashion, they must learn how to recognize and utilize underspecified language. This thesis argues that …

Scalable Pythagorean Mean-Based Incident Detection In Smart Transportation Systems, Md Jaminur Islam, Jose Paolo Talusan, Shameek Bhattacharjee, Francis Tiausas, Abhishek Dubey, Keiichi Yasumoto, Sajal K. Das

Computer Science Faculty Research & Creative Works

Modern smart cities need smart transportation solutions to quickly detect various traffic emergencies and incidents in the city to avoid cascading traffic disruptions. to materialize this, roadside units and ambient transportation sensors are being deployed to collect speed data that enables the monitoring of traffic conditions on each road segment. in this article, we first propose a scalable data-driven anomaly-Based traffic incident detection framework for a city-scale smart transportation system. Specifically, we propose an incremental region growing approximation algorithm for optimal Spatio-temporal clustering of road segments and their data; such that road segments are strategically divided into highly correlated clusters. …

Improved Models Of Elastic Scheduling, Marion Baumli Sudvarg

McKelvey School of Engineering Graduate Student Theses & Dissertations

In real-time computing systems, \textit{timely} execution is a requirement of \textit{correct} execution. Such systems are widely found in robotics and autonomous vehicle applications, mobile spectrometry of atmospheric aerosols, real-time hybrid simulation for natural hazards engineering, and in prompt localization of transients such as gamma-ray bursts for time-domain and multi-messenger astrophysics. \textit{Elastic scheduling} provides a framework to adjust computational rates and workloads in systems for which timeliness cannot otherwise be guaranteed. While originally proposed for periodic tasks executing on a single processor, elastic scheduling has since been extended to sequential and parallel execution on multiple processors and to earliest deadline first …