Open Access. Powered by Scholars. Published by Universities.®
Artificial Intelligence and Robotics Commons™
Open Access. Powered by Scholars. Published by Universities.®
- Keyword
-
- Software engineering (6)
- Machine learning (4)
- ChatGPT (3)
- Semantics (3)
- AI model generation (2)
-
- Cross-validation (2)
- Deep Learning (2)
- Deep learning (2)
- Deep learning models (2)
- Disambiguation (2)
- Generative AI (2)
- Large language model (2)
- Mobile Computing (2)
- Mobile crowdsourcing (2)
- Open-source software (2)
- Search trajectory (2)
- Sentiment analysis (2)
- Sequence alignment (2)
- Software Q&A (2)
- Specification (2)
- Test case generation (2)
- 3D Point Cloud (1)
- AI frameworks (1)
- AI libraries (1)
- AI4SE (1)
- API linking (1)
- Academic AI repository (1)
- Activity Recognition (1)
- Additive manufacturing (1)
- Adversarial patch attack (1)
Articles 1 - 30 of 67
Full-Text Articles in Artificial Intelligence and Robotics
Exploring The Potential Of Chatgpt In Automated Code Refinement: An Empirical Study, Qi Guo, Shangqing Liu, Junming Cao, Xiaohong Li, Xin Peng, Xiaofei Xie, Bihuan Chen
Exploring The Potential Of Chatgpt In Automated Code Refinement: An Empirical Study, Qi Guo, Shangqing Liu, Junming Cao, Xiaohong Li, Xin Peng, Xiaofei Xie, Bihuan Chen
Research Collection School Of Computing and Information Systems
Code review is an essential activity for ensuring the quality and maintainability of software projects. However, it is a time-consuming and often error-prone task that can significantly impact the development process. Recently, ChatGPT, a cutting-edge language model, has demonstrated impressive performance in various natural language processing tasks, suggesting its potential to automate code review processes. However, it is still unclear how well ChatGPT performs in code review tasks. To fill this gap, in this paper, we conduct the first empirical study to understand the capabilities of ChatGPT in code review tasks, specifically focusing on automated code refinement based on given …
Delving Into Multimodal Prompting For Fine-Grained Visual Classification, Xin Jiang, Hao Tang, Junyao Gao, Xiaoyu Du, Shengfeng He, Zechao Li
Delving Into Multimodal Prompting For Fine-Grained Visual Classification, Xin Jiang, Hao Tang, Junyao Gao, Xiaoyu Du, Shengfeng He, Zechao Li
Research Collection School Of Computing and Information Systems
Fine-grained visual classification (FGVC) involves categorizing fine subdivisions within a broader category, which poses challenges due to subtle inter-class discrepancies and large intra-class variations. However, prevailing approaches primarily focus on uni-modal visual concepts. Recent advancements in pre-trained vision-language models have demonstrated remarkable performance in various high-level vision tasks, yet the applicability of such models to FGVC tasks remains uncertain. In this paper, we aim to fully exploit the capabilities of cross-modal description to tackle FGVC tasks and propose a novel multimodal prompting solution, denoted as MP-FGVC, based on the contrastive language-image pertaining (CLIP) model. Our MP-FGVC comprises a multimodal prompts …
Conversational Localization: Indoor Human Localization Through Intelligent Conversation, Sheshadri Smitha, Kotaro Hara
Conversational Localization: Indoor Human Localization Through Intelligent Conversation, Sheshadri Smitha, Kotaro Hara
Research Collection School Of Computing and Information Systems
We propose a novel sensorless approach to indoor localization by leveraging natural language conversations with users, which we call conversational localization. To show the feasibility of conversational localization, we develop a proof-of-concept system that guides users to describe their surroundings in a chat and estimates their position based on the information they provide. We devised a modular architecture for our system with four modules. First, we construct an entity database with available image-based floor maps. Second, we enable the dynamic identification and scoring of information provided by users through our utterance processing module. Then, we implement a conversational agent that …
A Reliable And Secure Mobile Cyber-Physical Digital Microfluidic Biochip For Intelligent Healthcare, Yinan Yao, Decheng Qiu, Huangda Liu, Zhongliao Yang, Ximeng Liu, Yang Yang, Chen Dong
A Reliable And Secure Mobile Cyber-Physical Digital Microfluidic Biochip For Intelligent Healthcare, Yinan Yao, Decheng Qiu, Huangda Liu, Zhongliao Yang, Ximeng Liu, Yang Yang, Chen Dong
Research Collection School Of Computing and Information Systems
Digital microfluidic, as an emerging and potential technology, diversifies the biochemical applications platform, such as protein dilution sewage detection. At present, a vast majority of universal cyberphysical digital microfluidic biochips (DMFBs) transmit data through wires via personal computers and microcontrollers (like Arduino), consequently, susceptible to various security threats and with the popularity of wireless devices, losing competitiveness gradually. On the premise that security be ensured first and foremost, calls for wireless portable, safe, and economical DMFBs are imperative to expand their application fields, engage more users, and cater to the trend of future wireless communication. To this end, a new …
Dexbert: Effective, Task-Agnostic And Fine-Grained Representation Learning Of Android Bytecode, Tiezhu Sun, Kevin Allix, Kisub Kim, Xin Zhou, Dongsun Kim, David Lo, Tegawendé F. Bissyande, Jacques Klein
Dexbert: Effective, Task-Agnostic And Fine-Grained Representation Learning Of Android Bytecode, Tiezhu Sun, Kevin Allix, Kisub Kim, Xin Zhou, Dongsun Kim, David Lo, Tegawendé F. Bissyande, Jacques Klein
Research Collection School Of Computing and Information Systems
The automation of an increasingly large number of software engineering tasks is becoming possible thanks to Machine Learning (ML). One foundational building block in the application of ML to software artifacts is the representation of these artifacts ( e.g. , source code or executable code) into a form that is suitable for learning. Traditionally, researchers and practitioners have relied on manually selected features, based on expert knowledge, for the task at hand. Such knowledge is sometimes imprecise and generally incomplete. To overcome this limitation, many studies have leveraged representation learning, delegating to ML itself the job of automatically devising suitable …
Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, Xuan-Bach Dinh Le, David Lo
Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, Xuan-Bach Dinh Le, David Lo
Research Collection School Of Computing and Information Systems
Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the answers generated by ChatGPT are of low quality. To verify this, we conduct a comparative evaluation of human-written and ChatGPT-generated answers. Our methodology employs both automatic comparison and a manual study. Our results suggest that human-written and ChatGPT-generated answers are semantically similar, however, human-written answers outperform ChatGPT-generated ones consistently across multiple aspects, …
Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le-Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, David Lo, David Lo
Are We Ready To Embrace Generative Ai For Software Q&A?, Bowen Xu, Thanh-Dat Nguyen, Thanh Le-Cong, Thong Hoang, Jiakun Liu, Kisub Kim, Chen Gong, Changan Niu, Chenyu Wang, David Lo, David Lo
Research Collection School Of Computing and Information Systems
Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the answers generated by ChatGPT are of low quality. To verify this, we conduct a comparative evaluation of human-written and ChatGPT-generated answers. Our methodology employs both automatic comparison and a manual study. Our results suggest that human-written and ChatGPT-generated answers are semantically similar, however, human-written answers outperform ChatGPT-generated ones consistently across multiple aspects, …
Automated Question Title Reformulation By Mining Modifcation Logs From Stack Overflow, Ke Liu, Xiang Chen, Chunyang Chen, Xiaofei Xie, Zhanqi Cui
Automated Question Title Reformulation By Mining Modifcation Logs From Stack Overflow, Ke Liu, Xiang Chen, Chunyang Chen, Xiaofei Xie, Zhanqi Cui
Research Collection School Of Computing and Information Systems
In Stack Overflow, developers may not clarify and summarize the critical problems in the question titles due to a lack of domain knowledge or poor writing skills. Previous studies mainly focused on automatically generating the question titles by analyzing the posts’ problem descriptions and code snippets. In this study, we aim to improve title quality from the perspective of question title reformulation and propose a novel approach QETRA motivated by the findings of our formative study. Specifically, by mining modification logs from Stack Overflow, we first extract title reformulation pairs containing the original title and the reformulated title. Then we …
Semantic-Based Neural Network Repair, Richard Schumi, Jun Sun
Semantic-Based Neural Network Repair, Richard Schumi, Jun Sun
Research Collection School Of Computing and Information Systems
Recently, neural networks have spread into numerous fields including many safety-critical systems. Neural networks are built (and trained) by programming in frameworks such as TensorFlow and PyTorch. Developers apply a rich set of pre-defined layers to manually program neural networks or to automatically generate them (e.g., through AutoML). Composing neural networks with different layers is error-prone due to the non-trivial constraints that must be satisfied in order to use those layers. In this work, we propose an approach to automatically repair erroneous neural networks. The challenge is in identifying a minimal modification to the network so that it becomes valid. …
Techsumbot: A Stack Overflow Answer Summarization Tool For Technical Query, Chengran Yang, Bowen Xu, Jiakun Liu, David Lo
Techsumbot: A Stack Overflow Answer Summarization Tool For Technical Query, Chengran Yang, Bowen Xu, Jiakun Liu, David Lo
Research Collection School Of Computing and Information Systems
Stack Overflow is a popular platform for developers to seek solutions to programming-related problems. However, prior studies identified that developers may suffer from the redundant, useless, and incomplete information retrieved by the Stack Overflow search engine. To help developers better utilize the Stack Overflow knowledge, researchers proposed tools to summarize answers to a Stack Overflow question. However, existing tools use hand-craft features to assess the usefulness of each answer sentence and fail to remove semantically redundant information in the result. Besides, existing tools only focus on a certain programming language and cannot retrieve up-to-date new posted knowledge from Stack Overflow. …
Trustworthy And Synergistic Artificial Intelligence For Software Engineering: Vision And Roadmaps, David Lo
Trustworthy And Synergistic Artificial Intelligence For Software Engineering: Vision And Roadmaps, David Lo
Research Collection School Of Computing and Information Systems
For decades, much software engineering research has been dedicated to devising automated solutions aimed at enhancing developer productivity and elevating software quality. The past two decades have witnessed an unparalleled surge in the development of intelligent solutions tailored for software engineering tasks. This momentum established the Artificial Intelligence for Software Engineering (AI4SE) area, which has swiftly become one of the most active and popular areas within the software engiueering field. This Future of Software Engineering (FoSE) paper navigates through several focal points. It commences with a succinct introduction and history of AI4SE. Thereafter, it underscores the core challenges inherent to …
What Do Users Ask In Open-Source Ai Repositories? An Empirical Study Of Github Issues, Zhou Yang, Chenyu Wang, Jieke Shi, Thong Hoang, Pavneet Singh Kochhar, Qinghua Lu, Zhenchang Xing, David Lo
What Do Users Ask In Open-Source Ai Repositories? An Empirical Study Of Github Issues, Zhou Yang, Chenyu Wang, Jieke Shi, Thong Hoang, Pavneet Singh Kochhar, Qinghua Lu, Zhenchang Xing, David Lo
Research Collection School Of Computing and Information Systems
Artificial Intelligence (AI) systems, which benefit from the availability of large-scale datasets and increasing computational power, have become effective solutions to various critical tasks, such as natural language understanding, speech recognition, and image processing. The advancement of these AI systems is inseparable from open-source software (OSS). Specifically, many benchmarks, implementations, and frameworks for constructing AI systems are made open source and accessible to the public, allowing researchers and practitioners to reproduce the reported results and broaden the application of AI systems. The development of AI systems follows a data-driven paradigm and is sensitive to hyperparameter settings and data separation. Developers …
Bubbleu: Exploring Augmented Reality Game Design With Uncertain Ai-Based Interaction, Minji Kim, Kyungjin Lee, Rajesh Krishna Balan, Youngki Lee
Bubbleu: Exploring Augmented Reality Game Design With Uncertain Ai-Based Interaction, Minji Kim, Kyungjin Lee, Rajesh Krishna Balan, Youngki Lee
Research Collection School Of Computing and Information Systems
Object detection, while being an attractive interaction method for Augmented Reality (AR), is fundamentally error-prone due to the probabilistic nature of the underlying AI models, resulting in sub-optimal user experiences. In this paper, we explore the effect of three game design concepts, Ambiguity, Transparency, and Controllability, to provide better gameplay experiences in AR games that use error-prone object detection-based interaction modalities. First, we developed a base AR pet breeding game, called Bubbleu that uses object detection as a key interaction method. We then implemented three different variants, each according to the three concepts, to investigate the impact of each design …
Design, Development, And Evaluation Of An Interactive Personalized Social Robot To Monitor And Coach Post-Stroke Rehabilitation Exercises, Min Hun Lee, Daniel P. Siewiorek, Asim Smailagic, Alexandre Bernardino, Sergi Bermudez I Badia
Design, Development, And Evaluation Of An Interactive Personalized Social Robot To Monitor And Coach Post-Stroke Rehabilitation Exercises, Min Hun Lee, Daniel P. Siewiorek, Asim Smailagic, Alexandre Bernardino, Sergi Bermudez I Badia
Research Collection School Of Computing and Information Systems
Socially assistive robots are increasingly being explored to improve the engagement of older adults and people with disability in health and well-being-related exercises. However, even if people have various physical conditions, most prior work on social robot exercise coaching systems has utilized generic, predefined feedback. The deployment of these systems still remains a challenge. In this paper, we present our work of iteratively engaging therapists and post-stroke survivors to design, develop, and evaluate a social robot exercise coaching system for personalized rehabilitation. Through interviews with therapists, we designed how this system interacts with the user and then developed an interactive …
The Vehicle Routing Problem With Simultaneous Pickup And Delivery And Occasional Drivers, Vincent F. Yu, Grace Aloina, Panca Jodiawan, Aldy Gunawan, Tsung-C. Huang
The Vehicle Routing Problem With Simultaneous Pickup And Delivery And Occasional Drivers, Vincent F. Yu, Grace Aloina, Panca Jodiawan, Aldy Gunawan, Tsung-C. Huang
Research Collection School Of Computing and Information Systems
This research addresses the Vehicle Routing Problem with Simultaneous Pickup and Delivery and Occasional Drivers (VRPSPDOD), which is inspired from the importance of addressing product returns and the emerging notion of involving available crowds to perform pickup and delivery activities in exchange for some compensation. At the depot, a set of regular vehicles is available to deliver and/or pick up customers’ goods. A set of occasional drivers, each defined by their origin, destination, and flexibility, is also able to help serve the customers. The objective of VRPSPDOD is to minimize the total traveling cost of operating regular vehicles and total …
I Know What You Are Searching For: Code Snippet Recommendation From Stack Overflow Posts, Zhipeng Gao, Xin Xia, David Lo, John C. Grundy, Xindong Zhang, Zhenchang Xing
I Know What You Are Searching For: Code Snippet Recommendation From Stack Overflow Posts, Zhipeng Gao, Xin Xia, David Lo, John C. Grundy, Xindong Zhang, Zhenchang Xing
Research Collection School Of Computing and Information Systems
Stack Overflow has been heavily used by software developers to seek programming-related information. More and more developers use Community Question and Answer forums, such as Stack Overflow, to search for code examples of how to accomplish a certain coding task. This is often considered to be more efficient than working from source documentation, tutorials, or full worked examples. However, due to the complexity of these online Question and Answer forums and the very large volume of information they contain, developers can be overwhelmed by the sheer volume of available information. This makes it hard to find and/or even be aware …
Biasfinder: Metamorphic Test Generation To Uncover Bias For Sentiment Analysis Systems, Muhammad Hilmi Asyrofi, Zhou Yang, Imam Nur Bani Yusuf, Hong Jin Kang, Thung Ferdian, David Lo
Biasfinder: Metamorphic Test Generation To Uncover Bias For Sentiment Analysis Systems, Muhammad Hilmi Asyrofi, Zhou Yang, Imam Nur Bani Yusuf, Hong Jin Kang, Thung Ferdian, David Lo
Research Collection School Of Computing and Information Systems
Artificial intelligence systems, such as Sentiment Analysis (SA) systems, typically learn from large amounts of data that may reflect human bias. Consequently, such systems may exhibit unintended demographic bias against specific characteristics (e.g., gender, occupation, country-of-origin, etc.). Such bias manifests in an SA system when it predicts different sentiments for similar texts that differ only in the characteristic of individuals described. To automatically uncover bias in SA systems, this paper presents BiasFinder, an approach that can discover biased predictions in SA systems via metamorphic testing. A key feature of BiasFinder is the automatic curation of suitable templates from any given …
Recipegen++: An Automated Trigger Action Programs Generator, Imam Nur Bani Yusuf, Diyanah Abdul Jamal, Lingxiao Jiang, David Lo
Recipegen++: An Automated Trigger Action Programs Generator, Imam Nur Bani Yusuf, Diyanah Abdul Jamal, Lingxiao Jiang, David Lo
Research Collection School Of Computing and Information Systems
Trigger Action Programs (TAPs) are event-driven rules that allow users to automate smart-devices and internet services. Users can write TAPs by specifying triggers and actions from a set of predefined channels and functions. Despite its simplicity, composing TAPs can still be challenging for users due to the enormous search space of available triggers and actions. The growing popularity of TAPs is followed by the increasing number of supported devices and services, resulting in a huge number of possible combinations between triggers and actions. Motivated by such a fact, we improve our prior work and propose RecipeGen++, a deep-learning-based approach that …
Answer Summarization For Technical Queries: Benchmark And New Approach, Chengran Yang, Bowen Xu, Ferdian Thung, Yucen Shi, Ting Zhang, Zhou Yang, Xin Zhou, Jieke Shi, Junda He, Donggyun Han, David Lo
Answer Summarization For Technical Queries: Benchmark And New Approach, Chengran Yang, Bowen Xu, Ferdian Thung, Yucen Shi, Ting Zhang, Zhou Yang, Xin Zhou, Jieke Shi, Junda He, Donggyun Han, David Lo
Research Collection School Of Computing and Information Systems
Prior studies have demonstrated that approaches to generate an answer summary for a given technical query in Software Question and Answer (SQA) sites are desired. We find that existing approaches are assessed solely through user studies. Hence, a new user study needs to be performed every time a new approach is introduced; this is time-consuming, slows down the development of the new approach, and results from different user studies may not be comparable to each other. There is a need for a benchmark with ground truth summaries as a complement assessment through user studies. Unfortunately, such a benchmark is non-existent …
Physical Adversarial Attack On A Robotic Arm, Yifan Jia, Christopher M. Poskitt, Jun Sun, Sudipta Chattopadhyay
Physical Adversarial Attack On A Robotic Arm, Yifan Jia, Christopher M. Poskitt, Jun Sun, Sudipta Chattopadhyay
Research Collection School Of Computing and Information Systems
Collaborative Robots (cobots) are regarded as highly safety-critical cyber-physical systems (CPSs) owing to their close physical interactions with humans. In settings such as smart factories, they are frequently augmented with AI. For example, in order to move materials, cobots utilize object detectors based on deep learning models. Deep learning, however, has been demonstrated as vulnerable to adversarial attacks: a minor change (noise) to benign input can fool the underlying neural networks and lead to a different result. While existing works have explored such attacks in the context of picture/object classification, less attention has been given to attacking neural networks used …
An Empirical Study On Data Distribution-Aware Test Selection For Deep Learning Enhancement, Qiang Hu, Yuejun Guo, Maxime Cordy, Xiaofei Xie, Lei Ma, Mike Papadakis, Yves Le Traon
An Empirical Study On Data Distribution-Aware Test Selection For Deep Learning Enhancement, Qiang Hu, Yuejun Guo, Maxime Cordy, Xiaofei Xie, Lei Ma, Mike Papadakis, Yves Le Traon
Research Collection School Of Computing and Information Systems
Similar to traditional software that is constantly under evolution, deep neural networks need to evolve upon the rapid growth of test data for continuous enhancement (e.g., adapting to distribution shift in a new environment for deployment). However, it is labor intensive to manually label all of the collected test data. Test selection solves this problem by strategically choosing a small set to label. Via retraining with the selected set, deep neural networks will achieve competitive accuracy. Unfortunately, existing selection metrics involve three main limitations: (1) using different retraining processes, (2) ignoring data distribution shifts, and (3) being insufficiently evaluated. To …
Test Mimicry To Assess The Exploitability Of Library Vulnerabilities, Hong Jin Kang, Truong Giang Nguyen, Bach Le, Corina S. Pasareanu, David Lo
Test Mimicry To Assess The Exploitability Of Library Vulnerabilities, Hong Jin Kang, Truong Giang Nguyen, Bach Le, Corina S. Pasareanu, David Lo
Research Collection School Of Computing and Information Systems
Modern software engineering projects often depend on open-source software libraries, rendering them vulnerable to potential security issues in these libraries. Developers of client projects have to stay alert of security threats in the software dependencies. While there are existing tools that allow developers to assess if a library vulnerability is reachable from a project, they face limitations. Call graphonly approaches may produce false alarms as the client project may not use the vulnerable code in a way that triggers the vulnerability, while test generation-based approaches faces difficulties in overcoming the intrinsic complexity of exploiting a vulnerability, where extensive domain knowledge …
Reinforcement Learning Approach To Solve Dynamic Bi-Objective Police Patrol Dispatching And Rescheduling Problem, Waldy Joe, Hoong Chuin Lau, Jonathan Pan
Reinforcement Learning Approach To Solve Dynamic Bi-Objective Police Patrol Dispatching And Rescheduling Problem, Waldy Joe, Hoong Chuin Lau, Jonathan Pan
Research Collection School Of Computing and Information Systems
Police patrol aims to fulfill two main objectives namely to project presence and to respond to incidents in a timely manner. Incidents happen dynamically and can disrupt the initially-planned patrol schedules. The key decisions to be made will be which patrol agent to be dispatched to respond to an incident and subsequently how to adapt the patrol schedules in response to such dynamically-occurring incidents whilst still fulfilling both objectives; which sometimes can be conflicting. In this paper, we define this real-world problem as a Dynamic Bi-Objective Police Patrol Dispatching and Rescheduling Problem and propose a solution approach that combines Deep …
Exais: Executable Ai Semantics, Richard Schumi, Jun Sun
Exais: Executable Ai Semantics, Richard Schumi, Jun Sun
Research Collection School Of Computing and Information Systems
Neural networks can be regarded as a new programming paradigm, i.e., instead of building ever-more complex programs through (often informal) logical reasoning in the programmers' mind, complex 'AI' systems are built by optimising generic neural network models with big data. In this new paradigm, AI frameworks such as TensorFlow and PyTorch play a key role, which is as essential as the compiler for traditional programs. It is known that the lack of a proper semantics for programming languages (such as C), i.e., a correctness specification for compilers, has contributed to many problematic program behaviours and security issues. While it is …
Benchmarking Library Recognition In Tweets, Ting Zhang, Divya Prabha Chandrasekaran, Ferdian Thung, David Lo
Benchmarking Library Recognition In Tweets, Ting Zhang, Divya Prabha Chandrasekaran, Ferdian Thung, David Lo
Research Collection School Of Computing and Information Systems
Software developers often use social media (such as Twitter) to shareprogramming knowledge such as new tools, sample code snippets,and tips on programming. One of the topics they talk about is thesoftware library. The tweets may contain useful information abouta library. A good understanding of this information, e.g., on thedeveloper’s views regarding a library can be beneficial to weigh thepros and cons of using the library as well as the general sentimentstowards the library. However, it is not trivial to recognize whethera word actually refers to a library or other meanings. For example,a tweet mentioning the word “pandas" may refer to …
Feasibility Studies In Indoor Localization Through Intelligent Conversation, Sheshadri Smitha, Linus Cheng, Kotaro Hara
Feasibility Studies In Indoor Localization Through Intelligent Conversation, Sheshadri Smitha, Linus Cheng, Kotaro Hara
Research Collection School Of Computing and Information Systems
We propose a model to achieve human localization in indoor environments through intelligent conversation between users and an agent. We investigated the feasibility of conversational localization by conducting two studies. First, we conducted a Wizard-of-Oz study with N = 7 participants and studied the feasibility of localizing users through conversation. We identified challenges posed by users’ language and behavior. Second, we collected N = 800 user descriptions of virtual indoor locations from N = 80 Amazon Mechanical Turk participants to analyze user language. We explored the effects of conversational agent behavior and observed that people describe indoor locations differently based …
Comai: Enabling Lightweight, Collaborative Intelligence By Retrofitting Vision Dnns, Kasthuri Jayarajah, Dhanuja Wanniarachchige, Tarek Abdelzaher, Archan Misra
Comai: Enabling Lightweight, Collaborative Intelligence By Retrofitting Vision Dnns, Kasthuri Jayarajah, Dhanuja Wanniarachchige, Tarek Abdelzaher, Archan Misra
Research Collection School Of Computing and Information Systems
While Deep Neural Network (DNN) models have transformed machine vision capabilities, their extremely high computational complexity and model sizes present a formidable deployment roadblock for AIoT applications. We show that the complexity-vs-accuracy-vs-communication tradeoffs for such DNN models can be significantly addressed via a novel, lightweight form of “collaborative machine intelligence” that requires only runtime changes to the inference process. In our proposed approach, called ComAI, the DNN pipelines of different vision sensors share intermediate processing state with one another, effectively providing hints about objects located within their mutually-overlapping Field-of-Views (FoVs). CoMAI uses two novel techniques: (a) a secondary shallow ML …
Riconv++: Effective Rotation Invariant Convolutions For 3d Point Clouds Deep Learning, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung
Riconv++: Effective Rotation Invariant Convolutions For 3d Point Clouds Deep Learning, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung
Research Collection School Of Computing and Information Systems
3D point clouds deep learning is a promising field of research that allows a neural network to learn features of point clouds directly, making it a robust tool for solving 3D scene understanding tasks. While recent works show that point cloud convolutions can be invariant to translation and point permutation, investigations of the rotation invariance property for point cloud convolution has been so far scarce. Some existing methods perform point cloud convolutions with rotation-invariant features, existing methods generally do not perform as well as translation-invariant only counterpart. In this work, we argue that a key reason is that compared to …
Adadeep: A Usage-Driven, Automated Deep Model Compression Framework For Enabling Ubiquitous Intelligent Mobiles, Sicong Liu, Junzhao Du, Kaiming Nan, Zimu Zhou, Hui Liu, Zhangyang Wang, Yingyan Lin
Adadeep: A Usage-Driven, Automated Deep Model Compression Framework For Enabling Ubiquitous Intelligent Mobiles, Sicong Liu, Junzhao Du, Kaiming Nan, Zimu Zhou, Hui Liu, Zhangyang Wang, Yingyan Lin
Research Collection School Of Computing and Information Systems
Recent breakthroughs in deep neural networks (DNNs) have fueled a tremendously growing demand for bringing DNN-powered intelligence into mobile platforms. While the potential of deploying DNNs on resource-constrained platforms has been demonstrated by DNN compression techniques, the current practice suffers from two limitations: 1) merely stand-alone compression schemes are investigated even though each compression technique only suit for certain types of DNN layers; and 2) mostly compression techniques are optimized for DNNs’ inference accuracy, without explicitly considering other application-driven system performance (e.g., latency and energy cost) and the varying resource availability across platforms (e.g., storage and processing capability). To this …
Building Action Sets In A Deep Reinforcement Learner, Yongzhao Wang, Arunesh Sinha, Sky C.H. Wang, Michael P. Wellman
Building Action Sets In A Deep Reinforcement Learner, Yongzhao Wang, Arunesh Sinha, Sky C.H. Wang, Michael P. Wellman
Research Collection School Of Computing and Information Systems
In many policy-learning applications, the agent may execute a set of actions at each decision stage. Choosing among an exponential number of alternatives poses a computational challenge, and even representing actions naturally expressed as sets can be a tricky design problem. Building upon prior approaches that employ deep neural networks and iterative construction of action sets, we introduce a reward-shaping approach to apportion reward to each atomic action based on its marginal contribution within an action set, thereby providing useful feedback for learning to build these sets. We demonstrate our method in two environments where action spaces are combinatorial. Experiments …