Artificial Intelligence and Robotics Commons^™

Text-Driven Video Prediction, Xue Song, Jingjing Chen, Bin Zhu, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Current video generation models usually convert signals indicating appearance and motion received from inputs (e.g., image and text) or latent spaces (e.g., noise vectors) into consecutive frames, fulfilling a stochastic generation process for the uncertainty introduced by latent code sampling. However, this generation pattern lacks deterministic constraints for both appearance and motion, leading to uncontrollable and undesirable outcomes. To this end, we propose a new task called Text-driven Video Prediction (TVP). Taking the first frame and text caption as inputs, this task aims to synthesize the following frames. Specifically, appearance and motion components are provided by the image and caption …

How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generation, Cen Zhang, Yaowen Zheng, Mingqiang Bai, Yeting Li, Wei Ma, Xiaofei Xie

Research Collection School Of Computing and Information Systems

Fuzz drivers are essential for library API fuzzing. However, automatically generating fuzz drivers is a complex task, as it demands the creation of high-quality, correct, and robust API usage code. An LLM-based (Large Language Model) approach for generating fuzz drivers is a promising area of research. Unlike traditional program analysis-based generators, this text-based approach is more generalized and capable of harnessing a variety of API usage information, resulting in code that is friendly for human readers. However, there is still a lack of understanding regarding the fundamental issues on this direction, such as its e ectiveness and potential challenges. To …

Genixer : Empowering Multimodal Large Language Models As A Powerful Data Generator, Henry Hengyuan Zhao, Pan Zhou, Mike Zheng Shou

Research Collection School Of Computing and Information Systems

Multimodal Large Language Models (MLLMs) demonstrate exceptional problem-solving capabilities, but few research studies aim to gauge the ability to generate visual instruction tuning data. This paper proposes to explore the potential of empowering MLLMs to generate data independently without relying on GPT-4. We introduce Genixer, a comprehensive data generation pipeline consisting of four key steps: (i) instruction data collection, (ii) instruction template design, (iii) empowering MLLMs, and (iv) data generation and filtering. Additionally, we outline two modes of data generation: task-agnostic and task-specific, enabling controllable output. We demonstrate that a synthetic VQA-like dataset trained with LLaVA1.5 enhances performance on 10 …

A Two-Stage Matheuristic For The Home Healthcare Routing And Scheduling Problem With Perishable Products, Aldy Gunawan, Nabila Yuraisyah Salsabila, Vincent F. Yu, Pham Kien Minh Nguyen

Research Collection School Of Computing and Information Systems

This study proposes a home healthcare routing and scheduling problem, where perishable products such as medicines, vaccines, or meals must be provided for some patients’ treatments. This problem is formulated as a mixed integer linear programming (MILP). A two-stage matheuristic is then developed as the solution approach. The first stage is a local search to solve the nurse routing problem, and the second stage is run as the relaxed MILP to solve the scheduling problem. The matheuristic is tested on newly generated instances and compared with the results of CPLEX. The proposed matheuristic is able to obtain CPLEX solutions within …

Machine Visual Perception For Autonomous Docking Maneuvers, Derek B. Worth

Theses and Dissertations

This dissertation presents a novel approach to autonomous docking using machine learning for visual perception, particularly during probe and drogue aerial refueling. Autonomous vehicles have become pervasive in both civilian and defense sectors, and their ability to interact with their surroundings and each other autonomously is critical for future operations. Traditional methods relying on signals or inertial sensors face significant limitations such as interference, jamming, and drift. This research focuses on developing a computer vision-based solution to overcome these limitations. A novel pipeline, termed relative vectoring, is introduced, which utilizes dual object detection and machine learning to estimate relative positions …

Enabling Emg-Based Silent Speech Transcription Through Speech-To-Text Transfer Learning, Alexander T. Garcia

Master's Theses

In recent years, advances in deep learning have allowed various forms of electrographic signals, such as electroencephalography (EEG) and electromyography (EMG), to be used as a viable form of input in artificial intelligence applications, particularly for applications in the medical field. One such topic that EMG inputs have been used is in silent speech interfaces, or devices capable of processing speech without an audio-based input. The goal of this thesis is to explore a novel method of training a machine learning model to be used for silent speech interface development: using transfer learning to leverage a pre-trained speech recognition model …

Performance Of 5 Prominent Large Language Models In Surgical Knowledge Evaluation: A Comparative Analysis, Adam M. Ostrovsky, Joshua R. Chen, Vishal N. Shah, Babak Abai

Department of Surgery Faculty Papers

No abstract provided.

Getting To The Point: Contrasting Directness And Warmth In Motivational Embodied Conversational Agents, Michael O'Mahony, Cathy Ennis, Robert Ross

Conference papers

Enhancing long-term engagement with conversational agents remains a significant challenge. Controlling the perceived warmth or directness of an agent’s personality through the style of its generated text could be used to increase user likeability. This paper reports an investigation of a Wizard-of-Oz (WoZ) mediated study of two variants of a motivational embodied conversational agent to measure user perception of and attitudes towards warmth in interaction style. Results show a significant effect of users preferring an agent with a "more direct" personality for this scenario, though this effect is in many ways nuanced.

Diabetes Technology Meeting 2023, Tiffany Tian, Rachel E Aaron, Ashley Y Dunova, Johan H Jendle, David Kerr, Eda Cengiz, Andjela Drincic, John C Pickup, Kong Y Chen, Naomi Schwartz, Douglas B Muchmore, Halis K Akturk, Carol J Levy, Signe Schmidt, Riccardo Bellazzi, Alan H B Wu, Elias K Spanakis, Bijan Najafi, James Geoffrey Chase, Jane Jeffrie Seley, David C Klonoff

Center on Aging Staff Publications

Diabetes Technology Society hosted its annual Diabetes Technology Meeting from November 1 to November 4, 2023. Meeting topics included digital health; metrics of glycemia; the integration of glucose and insulin data into the electronic health record; technologies for insulin pumps, blood glucose monitors, and continuous glucose monitors; diabetes drugs and analytes; skin physiology; regulation of diabetes devices and drugs; and data science, artificial intelligence, and machine learning. A live demonstration of a personalized carbohydrate dispenser for people with diabetes was presented.

Monocular Bev Perception Of Road Scenes Via Front-To-Top View Projection, Wenxi Liu, Qi Li, Weixiang Yang, Jiaxin Cai, Yuanhong Yu, Yuexin Ma, Shengfeng He, Jia Pan

Research Collection School Of Computing and Information Systems

HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to expensive sensors and time-consuming computation. Camera-based methods usually need to perform road segmentation and view transformation separately, which often causes distortion and missing content. To push the limits of the technology, we present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view given a front-view monocular image only. We propose a front-to-top view projection (FTVP) module, which takes the constraint of cycle consistency between views into account and makes full use of their correlation to strengthen …

An Empirical Study Of Static Analysis Tools For Secure Code Review, Wachiraphan Charoenwet, Patanamon Thongtanunam, Van-Thuan Pham, Christoph Treude

Research Collection School Of Computing and Information Systems

Early identification of security issues in software development is vital to minimize their unanticipated impacts. Code review is a widely used manual analysis method that aims to uncover security issues along with other coding issues in software projects. While some studies suggest that automated static application security testing tools (SASTs) could enhance security issue identification, there is limited understanding of SAST’s practical effectiveness in supporting secure code review. Moreover, most SAST studies rely on synthetic or fully vulnerable versions of the subject program, which may not accurately represent real-world code changes in the code review process. To address this gap, …

Federated Learning Systems For Mobile Sensing Data, Xiaopeng Jiang

Dissertations

Federated Learning (FL) has emerged as a new distributed Deep Learning (DL) paradigm that enables privacy-aware training and inference on mobile devices with help from the cloud. This dissertation presents a comprehensive exploration of FL with mobile sensing data, covering systems, applications, and optimizations.

First, a mobile-cloud FL system, FLSys, is designed to balance model performance with resource consumption, tolerate communication failures, and achieve scalability. In FLSys, different DL models with different FL aggregation methods can be trained and accessed concurrently by different apps. In addition, FLSys provides advanced privacy-preserving mechanisms and a common API for third-party app developers to …

Ai And Academic Integrity, Max Sparkman Research Instruction Librarian, Milne Library, Brandon West Head Of Research & Instruction, Milne Library

Artificial Intelligence, 2024-25

This short module introduces students to important concepts regarding the use of AI and academic integrity. Concepts covered include a brief overview of generative AI, whether or not their use is considered plagiarism, how to use generative AI tools responsibly, and potential use cases. The module ends with a quiz where students can apply concepts from the module to three scenarios.

Module: Ai And Value-Neutrality, Jonathan Auyer Ph.D., Department Of Philosophy

Artificial Intelligence, 2024-25

Artificial Intelligence is on the tips of everyone’s tongues these days – What exactly is it? What will can it be used for? What will it be used for in the future? What problems will it create or solve or exacerbate? This learning module aims to look at a specific facet of AI — the issue of value-neutrality — by having students look inward at capabilities necessary for human flourishing and then ask whether AI can cultivate (or inhibit) those capabilities. This will lead to a discussion of what values underlie AI and what this says about whether or not …

Chatgpt Can Write My Class Assignments, Right?: A Guided Classroom Activity For Teaching The Strengths And Weaknesses Of Generative-Ai Tools, Dr. Peter J. Kalenda Assistant Professor, Elementary Science & Math, School Of Education

Artificial Intelligence, 2024-25

This module is designed to support your undergraduate or graduate students with developing an understanding of:

• How to use generative-AI tools, like ChatGPT,
• The strengths and weaknesses of generative-AI tools for completing different tasks,
• How to design and revise a classroom presentation that is aligned with course topics and objectives by using generative-AI tools as a resource, and
• How to engage in prompt engineering and prompt revision for generative-AI tools.

Your students are being tasked with creating an engaging presentation they will deliver to their peers that aligns with the topics and learning objectives of your course. However, they will …

As Soon As [A]I Speak[S], Lytton Smith Professor Of Poetry, Department Of English

Artificial Intelligence, 2024-25

This guide facilitates the delivery of a two-workshop sequence allowing students to understand how an Artificial Intelligence (AI) Large Language Model (LLM) composes poetry, and to use that knowledge to compose their own poem for optional display in both electronic and physical form. Students may participate in either/both workshops, and in each will engage in reflective work to understand how AI impacts them and vice versa. This guide and curriculum aligns with the "Contemporary Global Challenges, Creativity and Innovation" Participation in a Global Society outcome within the Geneseo GLOBE Curriculum.

Creating A Business In 50 Minutes With Ai, Mark A. Rider Vanarsdale Chair In Entrepreneurship, School Of Business

Artificial Intelligence, 2024-25

This lesson plan explores using artificial intelligence (AI) to enhance the process of business idea generation and validation. Over a dynamic 50-minute workshop, students will engage with AI tools, particularly Microsoft's Copilot, to develop viable business concepts aligned with their personal interests and market needs.

The session begins with the Ikigai exercise, guiding students to identify their passions, strengths, and potential economic opportunities. Following this, students will use AI to engage in divergent thinking, generating a wide range of business ideas and refining them through iterative prompts until they find promising concepts.

In the validation phase, students will employ AI …

Using Ai In Higher Ed: Is It Cheating?, David Levy Associate Professor & Chair Of Philosophy

Artificial Intelligence, 2024-25

Students generate a course-based or college-wide policy regarding the use of Generative AI, based on assigned readings, discussion, practice using the tools on writing assignments.

Psychology And The Digital Everywhere: Artificial Intelligence, Cassie Van Stolk Assistant Professor Of Psychology, Department Of Psychology

Artificial Intelligence, 2024-25

This module within the PSYC 390: Psychology and the Digital Everywhere course investigates the the implications of AI on human experiences using a biopsychosocial lens. Topics covered include an exploration of AI as a tool versus as an autonomous mind, ethical considerations of AI usage, and the promises and pitfalls of AI as a tool within the field of psychology. This module aligns with the "Contemporary Global Challenges, Creativity and Innovation" Participation in a Global Society outcome within the Geneseo GLOBE Curriculum.

Heterogeneous Multi-Robot Person-Following In Constrained Environments, Ori A. Miller

Electronic Theses and Dissertations

Maintaining visibility of a person requires effective systems. Security cameras or ground robots might be ideal, but they often fail in uncontrolled or unknown environments. A single ground robot struggles to navigate and track an agent at the same time. This work addresses the challenge by developing a multi-robot system with a slow ground robot and an agile aerial robot. Three methods are evaluated: FORWARD-PF, RL-Person Following (RL), and a baseline closed-loop method. FORWARD-PF proved the most reliable, completing all nine paths and reaching targets nearly twice as fast as RL. Despite completing seven paths, RL faltered on complex tasks. …

Supervised Classification Modeling On Louisiana Medicaid Data: A Comparative Study, Yead Rahman

Master's Theses

This thesis systematically optimizes and compares state-of-the-art supervised classification models for Louisiana Medicaid data targeting clinical services, COVID-19 infection, and tobacco use. These target variables are critically important as they represent key health outcomes and behaviors among Medicaid enrollees in Louisiana, a population often characterized by poverty and limited access to education. This study applies advanced machine learning techniques to identify the best model for multinomial and binary classification tasks. These include models such as Logistic Regression, XGBoost, AdaBoost, Random Forest, Decision Tree, Artificial Neural Networks, and Naïve Bayes. Extensive tuning of the hyperparameters and optimization of each classifier were …

Robotic Odor Source Localization Using Vision And Olfaction Sensing, Sunzid Hassan

Master's Theses

Robotic Odor Source Localization (ROSL) technology allows autonomous agents like robots to find an odor source in unknown environments. A successful odor source location depends crucially on an effective navigation algorithm that directs the robot towards the odor source. This thesis is a combination of three projects. First, we detail development of a versatile multi-modal robotic platform for ROSL real-world ROSL experimentation and discussed real-world validation of a traditional olfactionbased ROSL algorithm. Secondly, we introduced vision in ROSL by proposing a fusion navigation algorithm that integrates deep-learning enabled vision and olfaction-based navigation. This hybrid approach tackles challenges such as turbulent …

Democratization Of Custom, High Quality Large Language Models, Pablo Lopez

College of Computing and Digital Media Dissertations

Large Language Models (LLMs) have shown exceptional performance in several natural language processing (NLP) tasks. Customizing LLMs boosts their performance in domain specific tasks but typically requires substantial resources and effort for training, such as supervised fine-tuning. This research proposes methods to achieve significant accuracy improvements given minimal resources, particularly focusing on open-ended question answering with a given piece of context. We utilize an LLM’s self-generated training data to fine-tune the LLM and partial fine-tuning with on-demand GPU to reduce practitioner training costs. The research shows that these methods give significant performance gains in a Retrieval Augmented Generation (RAG) based …

The Impact Of Model Variations On The Robustness Of Deep Learning Models In Adversarial Settings, Firuz Juraev, Mohammed Abuhamad, Simon S. Woo, George K. Thiruvathukal, Tamer Abuhmed

Computer Science: Faculty Publications and Other Works

Rapid advancements of deep learning are accelerating adoption in a wide variety of applications, including safety-critical applications such as self-driving vehicles, drones, robots, and surveillance systems. These advancements include applying variations of sophisticated techniques that improve the performance of models. However, such models are not immune to adversarial manipulations, which can cause the system to misbehave and remain unnoticed by experts. The frequency of modifications to existing deep learning models necessitates thorough analysis to determine the impact on models’ robustness. In this work, we present an experimental evaluation of the effects of model modifications on deep learning model robustness using …

Developing Green Design, Leading Green And Low-Carbon Society, Yongxiang Lu

Bulletin of Chinese Academy of Sciences (Chinese Version)

Green design refers to a design principle and method that comprehensively considers energy and resource conservation, emission reduction, and environmental impact during the product manufacturing and operation process in the product design stage. It strives to reduce greenhouse gas emissions throughout the entire product lifecycle. In the era of information networks, green design is supported by big data, AI network collaborative design, network detection and monitoring, etc. It involves selecting energy-saving processes, green and low-carbon materials, and optimizing product geometry design and surface treatment to achieve efficient resource utilization and minimize waste. For example, selecting environmentally friendly materials through networked …

Challenges And Recommendations For Building Open Source Innovation Ecosystem For Large-Models In China, Xin Wen, Chao Zhang, Rui Guo, Kaihua Chen, Ze Feng, Qigang Zhu

Bulletin of Chinese Academy of Sciences (Chinese Version)

Addressing the current technological development issues that constrain the development of China’s large-scale model industry is needed to promote the continuous prosperity and development of the industry and enhance its international competitiveness. The study analyzes the significance of the open-source innovation ecosystem for the development of large-scale models in China. Based on reviewing the international experience of constructing the open-source innovation ecosystem, it further dissects the problems and challenges faced by the construction of the open-source innovation ecosystem for large-scale models in China and puts forward targeted suggestions. The study finds that the open-source innovation ecosystem for large-scale models in …

Research And Practice Of Digital Institute And Its System Framework, Jianjun Yu, Yue Wang, Kongmin Wang, Zhuomin Shi

Bulletin of Chinese Academy of Sciences (Chinese Version)

In the era of the digital economy, digital scientific research and digital government are both crucial components, serving as application scenarios enabled by new information technologies. Currently, the new generation of information technologies, especially artificial intelligence and big data, is instrumental in modernizing governance at scientific research institutions within Chinese Academy of Sciences (CAS). They notably drive paradigm shifts in scientific activities and accelerate the digital transformation of these institutions. The digital system formed by the digital transformation of management processes in scientific activities at research institutes is defined as the digital institute. This study analyzes the impact of digital …

Research And Implications Of The Us Clean Energy Strategy, Lanchun Li, Qing Liu, Wei Chen, Yun Tang, Jun Chen

Bulletin of Chinese Academy of Sciences (Chinese Version)

As the world enters a new period of carbon neutrality, the US government is actively building a clean energy innovation ecosystem through both internal and external measures. Systematically tracking and in-depth analysis of the intent, structure, approach, and other characteristics of the new phase of the US clean energy strategy is of practical significance to Chinese energy revolution. The US focuses on the strategic objectives of science and technology innovation, energy security, and infrastructure, and has constructed an innovation ecology characterized by technology lists, planning blueprints, full-chain research, and innovative subjects from the perspective of whole-government coordination, cross-institutional decision-making, deep …

Technology Governance And Governance Technology: From Perspective Of Regulatory Research On Blockchain Digital Assets, Yikai Wu, Guoan Li

Bulletin of Chinese Academy of Sciences (Chinese Version)

The Outline of the 14th Five-Year Plan takes blockchain as one of the key industries of the digital economy, and a number of ministries and commissions have also made clear deployments to accelerate the innovative application of blockchain, promote the digital transformation of the industry, and promote the high-quality development of the economy and society in the policy documents related to the informatization of the industry. The regulation and governance of blockchain digital assets cannot be separated from the understanding and analysis of blockchain technology itself, and observing the development mechanism of blockchain digital assets from the scientific and technological …

Ai-Driven Innovation And Development In Manufacturing: An Overview Of Trends, Issues, And Suggestions, Rongping Mu, Jingjing Guo, Yuchen Li, Qiang Li, Chun Jiang, Ze Feng, Guowei Dong

Bulletin of Chinese Academy of Sciences (Chinese Version)

Artificial intelligence technology has become a key force in driving innovation and development in global manufacturing industries, and its application in production automation, intelligent management, and other areas has become an important trend for innovation and development of manufacturing in all countries. This study comprehensively analyzes the innovation and development trend driven by artificial intelligence in global manufacturing, outlines the policies and measures of major countries in promoting the innovation and development of the manufacturing industries driven by artificial intelligence. It also points out the current situation and problems of the manufacturing’s innovation and development driven by artificial intelligence in …