Theory and Algorithms Commons^™

Win: Weight-Decay-Integrated Nesterov Acceleration For Faster Network Training, Pan Zhou, Xingyu Xie, Zhouchen Lin, Kim-Chuan Toh, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Training deep networks on large-scale datasets is computationally challenging. This work explores the problem of “how to accelerate adaptive gradient algorithms in a general manner", and proposes an effective Weight-decay-Integrated Nesterov acceleration (Win) to accelerate adaptive algorithms. Taking AdamW and Adam as examples, per iteration, we construct a dynamical loss that combines the vanilla training loss and a dynamic regularizer inspired by proximal point method, and respectively minimize the first- and second-order Taylor approximations of dynamical loss to update variable. This yields our Win acceleration that uses a conservative step and an aggressive step to update, and linearly combines these …

Faster Rates For Compressed Federated Learning With Client-Variance Reduction, Haoyu Zhao, Konstantin Burlachenko, Zhize Li, Peter Richtarik

Research Collection School Of Computing and Information Systems

Due to the communication bottleneck in distributed and federated learning applications, algorithms using communication compression have attracted significant attention and are widely used in practice. Moreover, the huge number, high heterogeneity, and limited availability of clients result in high client -variance. This paper addresses these two issues together by proposing compressed and clientvariance reduced methods COFIG and FRECON. We prove an O( (1+\omega)3/2\surdN+ (1+\omega)N2/3 S\epsilon2 S\epsilon2 ) bound on the number of communication rounds of COFIG in the nonconvex setting, where N is the total number of clients, S is the number of clients participating in each round, \epsilon is …

Screening Through A Broad Pool: Towards Better Diversity For Lexically Constrained Text Generation, Changsen Yuan, Heyan Huang, Yixin Cao, Qianwen Cao

Research Collection School Of Computing and Information Systems

Lexically constrained text generation (CTG) is to generate text that contains given constrained keywords. However, the text diversity of existing models is still unsatisfactory. In this paper, we propose a lightweight dynamic refinement strategy that aims at increasing the randomness of inference to improve generation richness and diversity while maintaining a high level of fluidity and integrity. Our basic idea is to enlarge the number and length of candidate sentences in each iteration, and choose the best for subsequent refinement. On the one hand, different from previous works, which carefully insert one token between two words per action, we insert …

Active Discovering New Slots For Task-Oriented Conversation, Yuxia Wu, Tianhao Dai, Zhedong Zheng, Lizi Liao

Research Collection School Of Computing and Information Systems

Existing task-oriented conversational systems heavily rely on domain ontologies with pre-defined slots and candidate values. In practical settings, these prerequisites are hard to meet, due to the emerging new user requirements and ever-changing scenarios. To mitigate these issues for better interaction performance, there are efforts working towards detecting out-of-vocabulary values or discovering new slots under unsupervised or semi-supervised learning paradigms. However, overemphasizing on the conversation data patterns alone induces these methods to yield noisy and arbitrary slot results. To facilitate the pragmatic utility, real-world systems tend to provide a stringent amount of human labeling quota, which offers an authoritative way …

Conditional Neural Heuristic For Multiobjective Vehicle Routing Problems, Mingfeng Fan, Yaoxin Wu, Zhiguang Cao, Wen Song, Guillaume Sartoretti, Huan Liu, Guohua Wu

Research Collection School Of Computing and Information Systems

Existing neural heuristics for multiobjective vehicle routing problems (MOVRPs) are primarily conditioned on instance context, which failed to appropriately exploit preference and problem size, thus holding back the performance. To thoroughly unleash the potential, we propose a novel conditional neural heuristic (CNH) that fully leverages the instance context, preference, and size with an encoder–decoder structured policy network. Particularly, in our CNH, we design a dual-attention-based encoder to relate preferences and instance contexts, so as to better capture their joint effect on approximating the exact Pareto front (PF). We also design a size-aware decoder based on the sinusoidal encoding to explicitly …

Music Genre Classification Capabilities Of Enhanced Neural Network Architectures, Joshua Engelkes

Scholarly Horizons: University of Minnesota, Morris Undergraduate Journal

With the increase of digital music audio uploads, applications that deal with music information have been widely requested by streaming platforms. Automatic music genre classification is an important function of music recommendation and music search applications. Since the music genre categorization criteria continually shift, data-driven methods such as neural networks have been proven especially useful to music information retrieval. An enhanced CNN architecture, the Bottom-up Broadcast Neural Network, uses mel-spectrograms to push music data through a network where important low-level information is preserved. An enhanced RNN architecture, the Independent Recurrent Neural Network for Music Genre Classification, takes advantage of the …

Imitate The Good And Avoid The Bad: An Incremental Approach To Safe Reinforcement Learning, Minh Huy Hoang, Mai Anh Tien, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

A popular framework for enforcing safe actions in Reinforcement Learning (RL) is Constrained RL, where trajectory based constraints on expected cost (or other cost measures) are employed to enforce safety and more importantly these constraints are enforced while maximizing expected reward. Most recent approaches for solving Constrained RL convert the trajectory based cost constraint into a surrogate problem that can be solved using minor modifications to RL methods. A key drawback with such approaches is an over or underestimation of the cost constraint at each state. Therefore, we provide an approach that does not modify the trajectory based cost constraint …

Dl-Drl: A Double-Level Deep Reinforcement Learning Approach For Large-Scale Task Scheduling Of Multi-Uav, Xiao Mao, Guohua Wu, Mingfeng Fan, Zhiguang Cao, Witold Pedrycz

Research Collection School Of Computing and Information Systems

Exploiting unmanned aerial vehicles (UAVs) to execute tasks is gaining growing popularity recently. To address the underlying task scheduling problem, conventional exact and heuristic algorithms encounter challenges such as rapidly increasing computation time and heavy reliance on domain knowledge, particularly when dealing with large-scale problems. The deep reinforcement learning (DRL) based methods that learn useful patterns from massive data demonstrate notable advantages. However, their decision space will become prohibitively huge as the problem scales up, thus deteriorating the computation efficiency. To alleviate this issue, we propose a double-level deep reinforcement learning (DL-DRL) approach based on a divide and conquer framework …

Recommendations With Minimum Exposure Guarantees: A Post-Processing Framework, Ramon Lopes, Rodrigo Alves, Antoine Ledent, Rodrygo L. T. Santos, Marius Kloft

Research Collection School Of Computing and Information Systems

Relevance-based ranking is a popular ingredient in recommenders, but it frequently struggles to meet fairness criteria because social and cultural norms may favor some item groups over others. For instance, some items might receive lower ratings due to some sort of bias (e.g. gender bias). A fair ranking should balance the exposure of items from advantaged and disadvantaged groups. To this end, we propose a novel post-processing framework to produce fair, exposure-aware recommendations. Our approach is based on an integer linear programming model maximizing the expected utility while satisfying a minimum exposure constraint. The model has fewer variables than previous …

The Measure Of Efficiency And Effectiveness When Using Artificial Intelligence (Ai) In Radiology, Jordan Watts

Theses, Dissertations and Capstones

Introduction: The use of artificial intelligence in radiology has helped radiologists identify patterns and abnormalities in medical images to diagnose and treat patients. Deep learning and machine learning algorithms have been used to assist physicians in detecting features that are not noticeable to the human eye. The FDA has approved almost 400 AI algorithms for radiology and estimated that the market for AI in medical imaging would grow from $21.48 billion in 2018 to $264.85 billion in 2028.

Purpose of the Study: The purpose of this research was to evaluate the use of artificial intelligence in radiology to determine its …

Reinforcement Learning For Optimal Kicking Actions In Humanoid Robotics: Advancing Robotic Autonomy And Versatility, Suresh Dodda, Sathish Kumar Chintala, Sukender Reddy Mallreddy, Sharath Chandra Macha, Yashwanth Vasa, Sapan Bharadwaj Bonala, Navin Kamuni, Sujatha Alla

Engineering Management & Systems Engineering Faculty Publications

Acquiring the necessary skills to perform a work effectively and efficiently requires a significant investment of time and computing power. Previous applications of Reinforcement Learning (RL) for action optimization in humanoid robotics have shown how promising this technology is for moving robotics towards true autonomy and versatility. Therefore, this study offers the first use of RL to create an entirely optimal kicking action for the Alderbaran Nao robot. Kicking motions that were steady, precise, quick, and able to kick farther than any existing RoboCup squad were generated by optimizing for a multi-objective reward function. We demonstrate that the ideal kicking …

Advancing Deep Learning With Graph-Based Structural Insights: From Graph Classification To Semantic Segmentation, Xin Ma

Computer Science and Engineering Dissertations - Archive

Deep learning has profoundly transformed machine learning by offering sophisticated data representations, yet effectively incorporating structural information remains a challenge. Structural data, whether explicit or implicit, has the potential to significantly enhance the performance of deep learning tasks. This research investigates the benefits of structural information across three crucial tasks: classification, clustering, and segmentation. For explicit structural data, where inputs are directly represented as graphs, we investigate graph-level classification in brain connectivity networks. We introduce the Multi-resolution Edge Network (MENET), a novel framework designed to identify disease-specific connectomic benchmarks with high discriminatory power across diagnostic categories. MENET leverages graph-level representations …

An Algorithm Based On Priority Rules For Solving A Multi-Drone Routing Problem In Hazardous Waste Collection, Youssef Harrath Dr., Jihene Kaabi Dr.

Research & Publications

This research investigates the problem of assigning pre-scheduled trips to multiple drones to collect hazardous waste from different sites in the minimum time. Each drone is subject to essential restrictions: maximum flying capacity and recharge operation. The goal is to assign the trips to the drones so that the waste is collected in the minimum time. This is done if the total flying time is equally distributed among the drones. An algorithm was developed to solve the problem. The algorithm is based on two main ideas: sort the trips according to a given priority rule and assign the current trip …

Image-To-Mesh Conversion Method For Multi-Tissue Medical Image Computing Simulations, Fotis Drakopoulos, Yixun Liu, Kevin Garner, Nikos Chrisochoides

Computer Science Faculty Publications

Converting a three-dimensional medical image into a 3D mesh that satisfies both the quality and fidelity constraints of predictive simulations and image-guided surgical procedures remains a critical problem. Presented is an image-to-mesh conversion method called CBC3D. It first discretizes a segmented image by generating an adaptive Body-Centered Cubic mesh of high-quality elements. Next, the tetrahedral mesh is converted into a mixed element mesh of tetrahedra, pentahedra, and hexahedra to decrease element count while maintaining quality. Finally, the mesh surfaces are deformed to their corresponding physical image boundaries, improving the mesh’s fidelity. The deformation scheme builds upon the ITK open-source library …

Enhanced Skin Cancer Diagnosis Through Grid Search Algorithm-Optimized Deep Learning Models For Skin Lesion Analysis, Rudresh Pillai, Neha Sharma, Sheifali Gupta, Deepali Gupta, Sapna Juneja, Saurav Malik, Hong Qin, Mohammed S. Alqahtani, Amel Ksibi

Computer Science Faculty Publications

Skin cancer is a widespread and perilous disease that necessitates prompt and precise detection for successful treatment. This research introduces a thorough method for identifying skin lesions by utilizing sophisticated deep learning (DL) techniques. The study utilizes three convolutional neural networks (CNNs)-CNN1, CNN2, and CNN3-each assigned to a distinct categorization job. Task 1 involves binary classification to determine whether skin lesions are present or absent. Task 2 involves distinguishing between benign and malignant lesions. Task 3 involves multiclass classification of skin lesion images to identify the precise type of skin lesion from a set of seven categories. The most optimal …

Trading Cloud Computing Stocks Using Sma, Xianrong Zheng, Lingyu Li

Information Technology & Decision Sciences Faculty Publications

As cloud computing adoption becomes mainstream, the cloud services market offers vast profits. Moreover, serverless computing, the next stage of cloud computing, comes with huge economic potential. To capitalize on this trend, investors are interested in trading cloud stocks. As high-growth technology stocks, investing in cloud stocks is both rewarding and challenging. The research question here is how a trading strategy will perform on cloud stocks. As a result, this paper employs an effective method—Simple Moving Average (SMA)—to trade cloud stocks. To evaluate its performance, we conducted extensive experiments with real market data that spans over 23 years. Results show …

The Feasibility Of Motion Tracking Camera System For Magnetic Suspension Wind Tunnel Tests, Hisham M. Shehata, David Cox, Mark Schoenenberger, Colin Britcher, Eli Shellabarger, Timothy Schott, Brendan Mcgovern

Mechanical & Aerospace Engineering Faculty Publications

The Entry Systems Modeling (ESM) Program at NASA has actively participated in the re-development of the Magnetic Suspension Balance System (MSBS) at the six-inch subsonic wind tunnel at NASA Langley Research Center. This initiative aims to enhance the MSBS system's capabilities, enabling the testing of stingless entry vehicle models at supersonic speeds. To achieve this, control algorithms are required to ensure magnetic levitation control and stability for models during free-oscillation dynamic responses. Currently, the system relies on electromagnetic position sensors to provide real-time 3 degrees of freedom control of a rigid body. While this approach has proven successful for subsonic …

Kinodynamic Motion Planning For A System With Squid Dynamics, Logan E. Beaver, Cong Wei, Wei-Kuo Yen

Mechanical & Aerospace Engineering Faculty Publications

This paper introduces a path planning algorithm for a system with squid dynamics in a cluttered environment. We capture the complex interactions of fin, arms, and body patterning by analyzing experimental data collected from observing squid motion. We extract nine motion primitives to build the control sequence for a time-optimal trajectory. This task is formulated as a mixed-integer program, and we generate the minimum-time trajectory using a sample-based approach. Numerical simulations illustrate the efficacy of this strategy and motivate ongoing and future efforts to exploration of squid motion features, improvement of the modeling, and experimental demonstrations of the motion planning …

Graph Coloring Reconfiguration, Reem Mahmoud

Theses and Dissertations

Reconfiguration is the concept of moving between different solutions to a problem by transforming one solution into another using some prescribed transformation rule (move). Given two solutions s₁ and s₂ of a problem, reconfiguration asks whether there exists a sequence of moves which transforms s₁ into s₂. Reconfiguration is an area of research with many contributions towards various fields such as mathematics and computer science.
The k-coloring reconfiguration problem asks whether there exists a sequence of moves which transforms one k-coloring of a graph G into another. A move in this case is a type …

Hyperparameter Estimation For Sparse Bayesian Learning Models, Feng Yu, Lixin Shen, Guohui Song

Mathematics & Statistics Faculty Publications

Sparse Bayesian learning (SBL) models are extensively used in signal processing and machine learning for promoting sparsity through hierarchical priors. The hyperparameters in SBL models are crucial for the model’s performance, but they are often difficult to estimate due to the nonconvexity and the high-dimensionality of the associated objective function. This paper presents a comprehensive framework for hyperparameter estimation in SBL models, encompassing well-known algorithms such as the expectation-maximization, MacKay, and convex bounding algorithms. These algorithms are cohesively interpreted within an alternating minimization and linearization (AML) paradigm, distinguished by their unique linearized surrogate functions. Additionally, a novel algorithm within the …

Integrative Machine Learning Approaches For Enhanced Classification Of Genomic Sequences: A Next-Generation Sequencing Perspective, Sujatha Alla, Nagesh Bheesetty, Sai Gireesh Komaragiri, Prasanthi Chidipudi, Joshit Mohanty, Sathish Kumar Chintala, Jubin Thomas, Jayapal Vummadi, Hemanth Volikatla, Navin Kamuni

Engineering Management & Systems Engineering Faculty Publications

The advent of Next-Generation Sequencing (NGS) techniques has revolutionized genomic research by enabling the rapid sequencing of DNA and RNA. This data can be used for various applications, including genome sequencing, transcriptome profiling, metagenomics, and epigenetics studies. For this study, DNA classifier dataset was extracted from UCI repository of machine learning databases. This vast amount of genomic data necessitates the development of sophisticated machine learning (ML) models for effective classification and analysis. This study presents a comprehensive comparison of various ML models, including Support Vector Machines (SVM), Random Forests (RF), and Neural Networks (NNs), approaches, in classifying genomic data. We …

Data Science In Finance: Challenges And Opportunities, Xianrong Zheng, Elizabeth Gildea, Sheng Chai, Tongxiao Zhang, Shuxi Wang

Information Technology & Decision Sciences Faculty Publications

Data science has become increasingly popular due to emerging technologies, including generative AI, big data, deep learning, etc. It can provide insights from data that are hard to determine from a human perspective. Data science in finance helps to provide more personal and safer experiences for customers and develop cutting-edge solutions for a company. This paper surveys the challenges and opportunities in applying data science to finance. It provides a state-of-the-art review of financial technologies, algorithmic trading, and fraud detection. Also, the paper identifies two research topics. One is how to use generative AI in algorithmic trading. The other is …

Continuous-Variable Quantum Computation Of The O(3) Model In 1+1 Dimensions, Raghav G. Jha, Felix Ringer, George Siopsis, Shane Thompson

Physics Faculty Publications

We formulate the O(3) nonlinear sigma model in 1+1 dimensions as a limit of a three-component scalar field theory restricted to the unit sphere in the large squeezing limit. This allows us to describe the model in terms of the continuous-variable (CV) approach to quantum computing. We construct the ground state and excited states using the coupled-cluster Ansatz and find excellent agreement with the exact diagonalization results for a small number of lattice sites. We then present the simulation protocol for the time evolution of the model using CV gates and obtain numerical results using a photonic quantum simulator. We …

The Computational Search For Unidentified Central Configurations Of The Newtonian N-Body Problem, Hannah G. Havel

CURE Proceedings

The N-body problem is a field of study in mathematics and physics that involves predicting the motion of particles moving under their mutual gravitational attraction. It is vital in celestial mechanics, such as planning collision-free satellite orbit trajectories. When beginning to understand the N-body problem, we can start by looking at equal masses of these particles or celestial bodies. As particles move, their position and velocity change, both energy and angular momentum are conserved. Sets of constant energy and angular momentum, known as integral manifolds, are higher-dimensional figures that represent constraints of movement to a system. Integral manifolds are described …

Enhancing Cyber Resilience Through Traffic Generation Patterns In Complex Networks: A Study On Cascading Failures, Aymar Le Père Tchimwa Bouom, Jean-Pierre Lienou, Wilson Ejuh Geh, Frederica Nelson, Sachin Shetty, Charles Kamhoua

VMASC Publications

Network resilience is the capacity of a network to maintain and restore its fundamental operations during or after a failure. This paper investigates the resilience of communication networks with heterogeneous nodes, with host nodes that generate and receive packets and routers that only forward packets. We focus on how traffic generation patterns, defined as the distribution of data packet creation across hosts, affect network resilience. While previous studies identified optimal host placements that balance traffic loads and enhance network performance, this research explores how traffic generation patterns influence network resilience, particularly during cascading failures, where the failure of one node …

Adaptive Neh With Constrained Nearest Neighbor Subtours For The Electric Vehicle Routing Problem With Time Windows, Andrew Struthers

All Master's Theses

The development of electric vehicles is currently considered one of the most innovative areas in manufacturing. Largely driven by the desire to reduce greenhouse emissions, electric vehicles are seen as a viable alternative to internal combustion engine cars. Starting from consumer cars, a dedicated effort is being made to translate this into commercial vehicles for freight and delivery. This research introduces a novel adaptive Nawaz, Enscore, Ham (NEH) algorithm with constrained nearest neighbor subtour (NEH-NN). This algorithm is tested on the standard benchmark problems in literature and used as a seed solution for the Genetic Algorithm (GA). The performance and …

Monotone Ordinal Expert Knowledge Acquisition For Explanation Of Machine Learning Models, Harlow Huber

All Master's Theses

There are significant difficulties for the acceptance of black-box Machine Learning (ML) models by subject matter experts (SMEs) despite significant achievements of many black-box models. A promising way to address this problem is by building a trustable, qualitative, interpretable models for the task based on SME knowledge. Such qualitative models can work as qualitative explainers of black-box models or as sanity checks for them. For instance, the expert model can expect that two cases belong to different classes, but the black box model predicts that they are in the same class. In this thesis, qualitative models operate with ordinal attributes, …

Wakening Past Concepts Without Past Data: Class-Incremental Learning From Online Placebos, Yaoyao Liu, Yingying Li, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

Not forgetting old class knowledge is a key challenge for class-incremental learning (CIL) when the model continuously adapts to new classes. A common technique to address this is knowledge distillation (KD), which penalizes prediction inconsistencies between old and new models. Such prediction is made with almost new class data, as old class data is extremely scarce due to the strict memory limitation in CIL. In this paper, we take a deep dive into KD losses and find that "using new class data for KD"not only hinders the model adaption (for learning new classes) but also results in low efficiency for …

A Secure And Robust Knowledge Transfer Framework Via Stratified-Causality Distribution Adjustment In Intelligent Collaborative Services, Ju Jia, Siqi Ma, Lina Wang, Yang Liu, Robert H. Deng

Research Collection School Of Computing and Information Systems

The rapid development of device-edge-cloud collaborative computing techniques has actively contributed to the popularization and application of intelligent service models. The intensity of knowledge transfer plays a vital role in enhancing the performance of intelligent services. However, the existing knowledge transfer methods are mainly implemented through data fine-tuning and model distillation, which may cause the leakage of data privacy or model copyright in intelligent collaborative systems. To address this issue, we propose a secure and robust knowledge transfer framework through stratified-causality distribution adjustment (SCDA) for device-edge-cloud collaborative services. Specifically, a simple yet effective density-based estimation is first employed to obtain …

Applications Of Independent And Identically Distributed (Iid) Random Processes In Polarimetry And Climatology, Dan Kestner

Dissertations, Master's Theses and Master's Reports

The unifying theme of this thesis is the characterization of “perfect randomness,” i.e., independent and identically distributed (IID) stochastic processes as these are applied in physical science. Two specific and mathematically distinct applications are chosen: (i) Radar and optical polarimetry; (ii) Analysis of time series in meteorology. In (i), IID process of a special kind, namely, with a distribution defined by symmetry, is used to link its multivariate Gaussian density to uniformity on the Poincaré sphere. This “statistical ellipsometry” approach is then used to relate polarimetric mismatches or imbalances to ellipsometric variables and suitably chosen cross-correlation measures. In (ii), recently …