Artificial Intelligence and Robotics Commons^™

Context-Aware Code Recommendation In Intellij Idea, Shamsa Abid, Hamid Abdul Basit, Shafay Shamail

Research Collection School Of Computing and Information Systems

Developers spend a lot of time online, searching for code to help them implement their desired features. While code recommenders help improve developers’ productivity, there is currently no support for context-aware code recommendation for opportunistic code reuse on-the-go. Typical code recommendation systems provide recommendations against a search query, whereas a code recommender that supports opportunistic reuse can recommend related code snippets that represent features that the developer may want to implement next. In this paper, we present a novel Context-aware Feature-driven API usage-based Code Recommender (CA-FACER) tool, which is an Intellij IDEA plugin that leverages a developer’s development context to …

Recipegen++: An Automated Trigger Action Programs Generator, Imam Nur Bani Yusuf, Diyanah Abdul Jamal, Lingxiao Jiang, David Lo

Research Collection School Of Computing and Information Systems

Trigger Action Programs (TAPs) are event-driven rules that allow users to automate smart-devices and internet services. Users can write TAPs by specifying triggers and actions from a set of predefined channels and functions. Despite its simplicity, composing TAPs can still be challenging for users due to the enormous search space of available triggers and actions. The growing popularity of TAPs is followed by the increasing number of supported devices and services, resulting in a huge number of possible combinations between triggers and actions. Motivated by such a fact, we improve our prior work and propose RecipeGen++, a deep-learning-based approach that …

Supporting The Discovery, Reuse, And Validation Of Cybersecurity Requirements At The Early Stages Of The Software Development Lifecycle, Jessica Antonia Steinmann

Doctoral Dissertations and Master's Theses

The focus of this research is to develop an approach that enhances the elicitation and specification of reusable cybersecurity requirements. Cybersecurity has become a global concern as cyber-attacks are projected to cost damages totaling more than $10.5 trillion dollars by 2025. Cybersecurity requirements are more challenging to elicit than other requirements because they are nonfunctional requirements that requires cybersecurity expertise and knowledge of the proposed system. The goal of this research is to generate cybersecurity requirements based on knowledge acquired from requirements elicitation and analysis activities, to provide cybersecurity specifications without requiring the specialized knowledge of a cybersecurity expert, and …

Physical Adversarial Attack On A Robotic Arm, Yifan Jia, Christopher M. Poskitt, Jun Sun, Sudipta Chattopadhyay

Research Collection School Of Computing and Information Systems

Collaborative Robots (cobots) are regarded as highly safety-critical cyber-physical systems (CPSs) owing to their close physical interactions with humans. In settings such as smart factories, they are frequently augmented with AI. For example, in order to move materials, cobots utilize object detectors based on deep learning models. Deep learning, however, has been demonstrated as vulnerable to adversarial attacks: a minor change (noise) to benign input can fool the underlying neural networks and lead to a different result. While existing works have explored such attacks in the context of picture/object classification, less attention has been given to attacking neural networks used …

Transrepair: Context-Aware Program Repair For Compilation Errors, Xueyang Li, Shangqing Liu, Ruitao Feng, Guozhu Meng, Xiaofei Xie, Kai Chen, Yang Liu

Research Collection School Of Computing and Information Systems

Automatically fixing compilation errors can greatly raise the productivity of software development, by guiding the novice or AI programmers to write and debug code. Recently, learning-based program repair has gained extensive attention and became the state of-the-art in practice. But it still leaves plenty of space for improvement. In this paper, we propose an end-to-end solution TransRepair to locate the error lines and create the correct substitute for a C program simultaneously. Superior to the counterpart, our approach takes into account the context of erroneous code and diagnostic compilation feedback. Then we devise a Transformer-based neural network to learn the …

Answer Summarization For Technical Queries: Benchmark And New Approach, Chengran Yang, Bowen Xu, Ferdian Thung, Yucen Shi, Ting Zhang, Zhou Yang, Xin Zhou, Jieke Shi, Junda He, Donggyun Han, David Lo

Research Collection School Of Computing and Information Systems

Prior studies have demonstrated that approaches to generate an answer summary for a given technical query in Software Question and Answer (SQA) sites are desired. We find that existing approaches are assessed solely through user studies. Hence, a new user study needs to be performed every time a new approach is introduced; this is time-consuming, slows down the development of the new approach, and results from different user studies may not be comparable to each other. There is a need for a benchmark with ground truth summaries as a complement assessment through user studies. Unfortunately, such a benchmark is non-existent …

Deep Learning Edge Detection In Image Inpainting, Zheng Zheng

Electronic Theses, Projects, and Dissertations

In recent years, deep learning has grown rapidly, and it has been creatively implemented for various applications. In 2019, deep learning based EdgeConnect image inpainting algorithm came out and occupied a place in the image inpainting field. Unlike traditional image inpainting methods which mainly read and use the color information of the remaining part of the image to fill the missing regions of the image, EdgeConnect uses the innovative edge-first and color-next approach. It uses an edge detector to generate an edge map of an image with missing regions, then the missing edges are completed by an edge model, finally …

An Empirical Study On Data Distribution-Aware Test Selection For Deep Learning Enhancement, Qiang Hu, Yuejun Guo, Maxime Cordy, Xiaofei Xie, Lei Ma, Mike Papadakis, Yves Le Traon

Research Collection School Of Computing and Information Systems

Similar to traditional software that is constantly under evolution, deep neural networks need to evolve upon the rapid growth of test data for continuous enhancement (e.g., adapting to distribution shift in a new environment for deployment). However, it is labor intensive to manually label all of the collected test data. Test selection solves this problem by strategically choosing a small set to label. Via retraining with the selected set, deep neural networks will achieve competitive accuracy. Unfortunately, existing selection metrics involve three main limitations: (1) using different retraining processes, (2) ignoring data distribution shifts, and (3) being insufficiently evaluated. To …

Test Mimicry To Assess The Exploitability Of Library Vulnerabilities, Hong Jin Kang, Truong Giang Nguyen, Bach Le, Corina S. Pasareanu, David Lo

Research Collection School Of Computing and Information Systems

Modern software engineering projects often depend on open-source software libraries, rendering them vulnerable to potential security issues in these libraries. Developers of client projects have to stay alert of security threats in the software dependencies. While there are existing tools that allow developers to assess if a library vulnerability is reachable from a project, they face limitations. Call graphonly approaches may produce false alarms as the client project may not use the vulnerable code in a way that triggers the vulnerability, while test generation-based approaches faces difficulties in overcoming the intrinsic complexity of exploiting a vulnerability, where extensive domain knowledge …

Analysis Of Digital Image Segmentation Algorithms, Khalilov Sirojiddin

Karakalpak Scientific Journal

Ushbu maqolada zamonaviy axborot-kommunikatsiya texnologiyalaridan foydalanishni kengaytirish maqsadida raqamli tasvirni qayta ishlash usullari va algoritmlari tahlil qilinadi. Maqolada, shuningdek, raqamli tasvirni qayta ishlash, tasvirni segmentatsiyalash usullari, WaterShed, MeanShift, FloodFill, GrabCut algoritmlarining afzalliklari va kamchiliklari o'rganiladi.

Monofacial Vs Bifacial Solar Photovoltaic Systems In Snowy Environments, Koami Soulemane Hayibo, Aliaksei Petsiuk, Pierce Mayville, Laura Brown, Joshua M. Pearce

Electrical and Computer Engineering Publications

There has been a recent surge in interest in the more accurate snow loss estimates for solar photovoltaic (PV) systems as large-scale deployments move into northern latitudes. Preliminary results show bifacial modules may clear snow faster than monofacial PV. This study analyzes snow losses on these two types of systems using empirical hourly data including energy, solar irradiation and albedo, and open-source image processing methods from images of the arrays in a northern environment in the winter. Projection transformations based on reference anchor points and snowless ground truth images provide reliable masking and optical distortion correction with fixed surveillance cameras. …

Wildfire Risk Assessment Using Convolutional Neural Networks And Modis Climate Data, Sean F. Nesbit

Master's Theses

Wildfires burn millions of acres of land each year leading to the destruction of homes and wildland ecosystems while costing governments billions in funding. As climate change intensifies drought volatility across the Western United States, wildfires are likely to become increasingly severe. Wildfire risk assessment and hazard maps are currently employed by fire services, but can often be outdated. This paper introduces an image-based dataset using climate and wildfire data from NASA’s Moderate Resolution Imaging Spectroradiometer (MODIS). The dataset consists of 32 climate and topographical layers captured across 0.1 deg by 0.1 deg tiled regions in California and Nevada between …

Reinforcement Learning Approach To Solve Dynamic Bi-Objective Police Patrol Dispatching And Rescheduling Problem, Waldy Joe, Hoong Chuin Lau, Jonathan Pan

Research Collection School Of Computing and Information Systems

Police patrol aims to fulfill two main objectives namely to project presence and to respond to incidents in a timely manner. Incidents happen dynamically and can disrupt the initially-planned patrol schedules. The key decisions to be made will be which patrol agent to be dispatched to respond to an incident and subsequently how to adapt the patrol schedules in response to such dynamically-occurring incidents whilst still fulfilling both objectives; which sometimes can be conflicting. In this paper, we define this real-world problem as a Dynamic Bi-Objective Police Patrol Dispatching and Rescheduling Problem and propose a solution approach that combines Deep …

Challenges In Migrating Imperative Deep Learning Programs To Graph Execution: An Empirical Study, Tatiana Castro Vélez, Raffi Khatchadourian, Mehdi Bagherzadeh, Anita Raja

Publications and Research

Efficiency is essential to support responsiveness w.r.t. ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code that supports symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development tends to produce DL code that is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, less error-prone imperative DL frameworks encouraging eager execution have emerged at the expense of run-time performance. While hybrid approaches aim for the "best of both worlds," the challenges in applying them in the real world are largely unknown. We conduct a data-driven analysis of challenges—and resultant bugs—involved …

Challenges In Migrating Imperative Deep Learning Programs To Graph Execution: An Empirical Study, Tatiana Castro Vélez, Raffi Khatchadourian, Mehdi Bagherzadeh, Anita Raja

Publications and Research

Efficiency is essential to support responsiveness w.r.t. ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code that supports symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development tends to produce DL code that is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, less error-prone imperative DL frameworks encouraging eager execution have emerged at the expense of run-time performance. While hybrid approaches aim for the "best of both worlds," the challenges in applying them in the real world are largely unknown. We conduct a data-driven analysis of challenges—and resultant bugs—involved …

Real Time Call-Flagging System To Respond To Suicidal Ideation In Call Centers, Vishnu Menon, Joseph Carrigan, Charles Floeder, Thomas Walton, Devin Mcguire

Honors Program: Senior Projects (Public)

The 2021-2022 Signature Performance Design Studio team developed a live audio call-flagging system that enables faster responses and new response pathways to veteran crises by call service representatives and their management team. Using a custom made deep learning model, live audio streaming server, and Teams broadcasting add-on, the system empowers Signature Performance call service representatives to make quicker and more well informed decisions to provide veteran’s the best care possible.

Risk Gameplay Analysis Using Stochastic Beam Search, Jacob Gillenwater

Electronic Theses and Dissertations

Hasbro’s RISK, first published in 1959, is a complex multiplayer strategy game that has received little attention from the scientific community. Training artificial intelligence (AI) agents using stochastic beam search gives insight into effective strategy when playing RISK. A comprehensive analysis of the systems of play challenges preconceptions about good strategy in some areas of the game while reinforcing those preconceptions in others. This study applies stochastic beam search to discover optimal strategies in RISK. Results of the search show both support for and challenges to traditionally held positions about RISK gameplay. While stochastic beam search competently investigates gameplay on …

Feasibility Studies In Indoor Localization Through Intelligent Conversation, Sheshadri Smitha, Linus Cheng, Kotaro Hara

Research Collection School Of Computing and Information Systems

We propose a model to achieve human localization in indoor environments through intelligent conversation between users and an agent. We investigated the feasibility of conversational localization by conducting two studies. First, we conducted a Wizard-of-Oz study with N = 7 participants and studied the feasibility of localizing users through conversation. We identified challenges posed by users’ language and behavior. Second, we collected N = 800 user descriptions of virtual indoor locations from N = 80 Amazon Mechanical Turk participants to analyze user language. We explored the effects of conversational agent behavior and observed that people describe indoor locations differently based …

Benchmarking Library Recognition In Tweets, Ting Zhang, Divya Prabha Chandrasekaran, Ferdian Thung, David Lo

Research Collection School Of Computing and Information Systems

Software developers often use social media (such as Twitter) to shareprogramming knowledge such as new tools, sample code snippets,and tips on programming. One of the topics they talk about is thesoftware library. The tweets may contain useful information abouta library. A good understanding of this information, e.g., on thedeveloper’s views regarding a library can be beneficial to weigh thepros and cons of using the library as well as the general sentimentstowards the library. However, it is not trivial to recognize whethera word actually refers to a library or other meanings. For example,a tweet mentioning the word “pandas" may refer to …

Exais: Executable Ai Semantics, Richard Schumi, Jun Sun

Research Collection School Of Computing and Information Systems

Neural networks can be regarded as a new programming paradigm, i.e., instead of building ever-more complex programs through (often informal) logical reasoning in the programmers' mind, complex 'AI' systems are built by optimising generic neural network models with big data. In this new paradigm, AI frameworks such as TensorFlow and PyTorch play a key role, which is as essential as the compiler for traditional programs. It is known that the lack of a proper semantics for programming languages (such as C), i.e., a correctness specification for compilers, has contributed to many problematic program behaviours and security issues. While it is …

Warehouse And Logistics: Smart Picking With Vuzix Smart Glasses, Elise Hemink

Frameless

Vuzix is an industry leader in augmented reality (AR) technology. We provide innovative products to an array of industries, a few being defense, security, enterprise, and consumers. Our AR technology provides a perfect balance of engagement in the digital and real worlds thanks to their innovative optics, AI apps and 5G capability.

Creating A Virtual Reality Experience In Service To A Non-Profit Agency, Frank Deese, Susan Lakin, Isabelle Anderson

Frameless

In the summer of 2018, RIT Professors Susan Lakin and Frank Deese discussed with the principal officers of the Society for the Protection and Care of Children (SPCC) in Rochester how the new technology of Virtual Reality might be used to not only impart information to viewers, but generate empathy for those receiving services from the organization as well as those performing those services. Their ultimate goal was to create an experience that could be viewed with VR headsets at fundraising events and on a website using low-cost Google Cardboard.

Machine Learning-Oriented Predictive Maintenance (Pdm) Framework For Autonomous Vehicles (Avs): Adopting Blockchain For Pdm Solution, Md Jobair Hossain Faruk, Hossain Shahriar, Maria Valero

Symposium of Student Scholars

Autonomous Vehicles (AVs) refers to smart, connected and multimedia cars with technological megatrends of the fourth industrial revolution (Industry 4.0) and have gained huge strive in today's world. AVs adopt automated driving systems (ADS) technique that permits the vehicle to manage and control driving points without human drivers by utilizing advanced equipment including a combination of sensors, controllers, onboard computers, actuators, algorithms, and advanced software embedded in the different parts of the vehicle. These advanced sensors provide unique inputs to the ADS to generate a path from point A to point B. Ensuring the safety of sensors by limiting maintenance …

Comai: Enabling Lightweight, Collaborative Intelligence By Retrofitting Vision Dnns, Kasthuri Jayarajah, Dhanuja Wanniarachchige, Tarek Abdelzaher, Archan Misra

Research Collection School Of Computing and Information Systems

While Deep Neural Network (DNN) models have transformed machine vision capabilities, their extremely high computational complexity and model sizes present a formidable deployment roadblock for AIoT applications. We show that the complexity-vs-accuracy-vs-communication tradeoffs for such DNN models can be significantly addressed via a novel, lightweight form of “collaborative machine intelligence” that requires only runtime changes to the inference process. In our proposed approach, called ComAI, the DNN pipelines of different vision sensors share intermediate processing state with one another, effectively providing hints about objects located within their mutually-overlapping Field-of-Views (FoVs). CoMAI uses two novel techniques: (a) a secondary shallow ML …

Riconv++: Effective Rotation Invariant Convolutions For 3d Point Clouds Deep Learning, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

Research Collection School Of Computing and Information Systems

3D point clouds deep learning is a promising field of research that allows a neural network to learn features of point clouds directly, making it a robust tool for solving 3D scene understanding tasks. While recent works show that point cloud convolutions can be invariant to translation and point permutation, investigations of the rotation invariance property for point cloud convolution has been so far scarce. Some existing methods perform point cloud convolutions with rotation-invariant features, existing methods generally do not perform as well as translation-invariant only counterpart. In this work, we argue that a key reason is that compared to …

The Effect Of Using The Gamification Strategy On Academic Achievement And Motivation Towards Learning Problem-Solving Skills In Computer And Information Technology Course Among Tenth Grade Female Students, Mazyunah Almutairi, Prof. Ahmad Almassaad

International Journal for Research in Education

Abstract

This study aimed to identify the effect of using the gamification strategy on academic achievement and motivation towards learning problem-solving skills in computer and information technology course. A quasi-experimental method was adopted. The study population included tenth-grade female students in Al-Badi’ah schools in Riyadh. The sample consisted of 54 students divided into two equal groups: control group and experimental group. The study tools comprised an achievement test and the motivation scale. The results showed that there were statistically significant differences between the two groups in the academic achievement test in favor of the experimental group, with a large effect …

Using Neural Networks To Model Guitar Distortion, Caleb Koch, Scott Hawley, Andrew Fyfe

Science University Research Symposium (SURS)

Guitar players have been modifying their guitar tone with audio effects ever since the mid-20th century. Traditionally, these effects have been achieved by passing a guitar signal through a series of electronic circuits which modify the signal to produce the desired audio effect. With advances in computer technology, audio “plugins” have been created to produce audio effects digitally through programming algorithms. More recently, machine learning researchers have been exploring the use of neural networks to produce audio effects that yield strikingly similar results to their analog counterparts. Recurrent Neural Networks and Temporal Convolutional Networks have proven to be exceptional at …

Comparative Analysis Of Rgb-Based Eye-Tracking For Large-Scale Human-Machine Applications, Brett Thaman, Trung Cao

Posters-at-the-Capitol

Gaze tracking has become an established technology that enables using an individual’s gaze as an input signal to support a variety of applications in the context of Human-Computer Interaction. Gaze tracking primarily relies on sensing devices such as infrared (IR) cameras. Nevertheless, in the recent years, several attempts have been realized at detecting gaze by acquiring and processing images acquired from standard RGB cameras. Nowadays, there are only a few publicly available open-source libraries and they have not been tested extensively. In this paper, we present the result of a comparative analysis that studied a commercial eye-tracking device using IR …

Contrastive Learning For Unsupervised Auditory Texture Models, Christina Trexler

Computer Science and Computer Engineering Undergraduate Honors Theses

Sounds with a high level of stationarity, also known as sound textures, have perceptually relevant features which can be captured by stimulus-computable models. This makes texture-like sounds, such as those made by rain, wind, and fire, an appealing test case for understanding the underlying mechanisms of auditory recognition. Previous auditory texture models typically measured statistics from auditory filter bank representations, and the statistics they used were somewhat ad-hoc, hand-engineered through a process of trial and error. Here, we investigate whether a better auditory texture representation can be obtained via contrastive learning, taking advantage of the stationarity of auditory textures to …

Adadeep: A Usage-Driven, Automated Deep Model Compression Framework For Enabling Ubiquitous Intelligent Mobiles, Sicong Liu, Junzhao Du, Kaiming Nan, Zimu Zhou, Hui Liu, Zhangyang Wang, Yingyan Lin

Research Collection School Of Computing and Information Systems

Recent breakthroughs in deep neural networks (DNNs) have fueled a tremendously growing demand for bringing DNN-powered intelligence into mobile platforms. While the potential of deploying DNNs on resource-constrained platforms has been demonstrated by DNN compression techniques, the current practice suffers from two limitations: 1) merely stand-alone compression schemes are investigated even though each compression technique only suit for certain types of DNN layers; and 2) mostly compression techniques are optimized for DNNs’ inference accuracy, without explicitly considering other application-driven system performance (e.g., latency and energy cost) and the varying resource availability across platforms (e.g., storage and processing capability). To this …