Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
- Discipline
- Keyword
-
- Machine Learning (cs.LG) (6)
- Learning systems (5)
- Reinforcement learning (5)
- Machine learning (4)
- Simple++ (4)
-
- Computational linguistics (3)
- Deep learning (3)
- Internet of things (3)
- Wireless networks (3)
- Antennas (2)
- Artificial Intelligence (cs.AI) (2)
- Artificial intelligence (2)
- Auto encoders (2)
- Benchmark (2)
- Computer vision (2)
- Condition (2)
- Constrained optimization (2)
- Design (2)
- Distributed optimization (2)
- Federated learning (2)
- Health care (2)
- Learn+ (2)
- Object Detection (2)
- Optimization (2)
- Optimization and Control (math.OC) (2)
- Reconfigurable intelligent surface (2)
- Stochastic systems (2)
- 5G mobile communication systems (1)
- 5g-and-beyond/6g (1)
- Accelerated algorithms (1)
Articles 1 - 30 of 67
Full-Text Articles in Physical Sciences and Mathematics
A Hybrid Artificial Intelligence Model For Detecting Keratoconus, Zaid Abdi Alkareem Alyasseri, Ali H. Al-Timemy, Ammar Kamal Abasi, Alexandru Lavric, Husam Jasim Mohammed, Hidenori Takahashi, Jose Arthur Milhomens Filho, Mauro Campos, Rossen M. Hazarbassanov, Siamak Yousefi
A Hybrid Artificial Intelligence Model For Detecting Keratoconus, Zaid Abdi Alkareem Alyasseri, Ali H. Al-Timemy, Ammar Kamal Abasi, Alexandru Lavric, Husam Jasim Mohammed, Hidenori Takahashi, Jose Arthur Milhomens Filho, Mauro Campos, Rossen M. Hazarbassanov, Siamak Yousefi
Machine Learning Faculty Publications
Machine learning models have recently provided great promise in diagnosis of several ophthalmic disorders, including keratoconus (KCN). Keratoconus, a noninflammatory ectatic corneal disorder characterized by progressive cornea thinning, is challenging to detect as signs may be subtle. Several machine learning models have been proposed to detect KCN, however most of the models are supervised and thus require large well-annotated data. This paper proposes a new unsupervised model to detect KCN, based on adapted flower pollination algorithm (FPA) and the k-means algorithm. We will evaluate the proposed models using corneal data collected from 5430 eyes at different stages of KCN severity …
A Damped Newton Method Achieves Global O(1/K2) And Local Quadratic Convergence Rate, Slavomír Hanzely, Dmitry Kamzolov, Dmitry Pasechnyuk, Alexander Gasnikov, Peter Richtárik, Martin Takáč
A Damped Newton Method Achieves Global O(1/K2) And Local Quadratic Convergence Rate, Slavomír Hanzely, Dmitry Kamzolov, Dmitry Pasechnyuk, Alexander Gasnikov, Peter Richtárik, Martin Takáč
Machine Learning Faculty Publications
In this paper, we present the first stepsize schedule for Newton method resulting in fast global and local convergence guarantees. In particular, a) we prove an O (1/k2) global rate, which matches the state-of-the-art global rate of cubically regularized Newton method of Polyak and Nesterov (2006) and of regularized Newton method of Mishchenko (2021) and Doikov and Nesterov (2021), b) we prove a local quadratic rate, which matches the best-known local rate of second-order methods, and c) our stepsize formula is simple, explicit, and does not require solving any subproblem. Our convergence proofs hold under affine-invariance assumptions closely related to …
Automs: Automatic Model Selection For Novelty Detection With Error Rate Control, Yifan Zhang, Haiyan Jiang, Haojie Ren, Changliang Zou, Dejing Dou
Automs: Automatic Model Selection For Novelty Detection With Error Rate Control, Yifan Zhang, Haiyan Jiang, Haojie Ren, Changliang Zou, Dejing Dou
Machine Learning Faculty Publications
Given an unsupervised novelty detection task on a new dataset, how can we automatically select a “best” detection model while simultaneously controlling the error rate of the best model? For novelty detection analysis, numerous detectors have been proposed to detect outliers on a new unseen dataset based on a score function trained on available clean data. However, due to the absence of labeled anomalous data for model evaluation and comparison, there is a lack of systematic approaches that are able to select the “best” model/detector (i.e., the algorithm as well as its hyperparameters) and achieve certain error rate control simultaneously. …
Factored Adaptation For Non-Stationary Reinforcement Learning, Fan Feng, Biwei Huang, Kun Zhang, Sara Magliacane
Factored Adaptation For Non-Stationary Reinforcement Learning, Fan Feng, Biwei Huang, Kun Zhang, Sara Magliacane
Machine Learning Faculty Publications
Dealing with non-stationarity in environments (e.g., in the transition dynamics) and objectives (e.g., in the reward functions) is a challenging problem that is crucial in real-world applications of reinforcement learning (RL). While most current approaches model the changes as a single shared embedding vector, we leverage insights from the recent causality literature to model non-stationarity in terms of individual latent change factors, and causal graphs across different environments. In particular, we propose Factored Adaptation for Non-Stationary RL (FANS-RL), a factored adaption approach that learns jointly both the causal structure in terms of a factored MDP, and a factored representation of …
Independence Testing-Based Approach To Causal Discovery Under Measurement Error And Linear Non-Gaussian Models, Haoyue Dai, Peter Spirtes, Kun Zhang
Independence Testing-Based Approach To Causal Discovery Under Measurement Error And Linear Non-Gaussian Models, Haoyue Dai, Peter Spirtes, Kun Zhang
Machine Learning Faculty Publications
Causal discovery aims to recover causal structures generating the observational data. Despite its success in certain problems, in many real-world scenarios the observed variables are not the target variables of interest, but the imperfect measures of the target variables. Causal discovery under measurement error aims to recover the causal graph among unobserved target variables from observations made with measurement error. We consider a specific formulation of the problem, where the unobserved target variables follow a linear non-Gaussian acyclic model, and the measurement process follows the random measurement error model. Existing methods on this formulation rely on non-scalable over-complete independent component …
Rare Gems: Finding Lottery Tickets At Initialization, Kartik Sreenivasan, Jy Yong Sohn, Liu Yang, Matthew Grinde, Alliot Nagle, Hongyi Wang, Eric Xing, Kangwook Lee, Dimitris Papailiopoulos
Rare Gems: Finding Lottery Tickets At Initialization, Kartik Sreenivasan, Jy Yong Sohn, Liu Yang, Matthew Grinde, Alliot Nagle, Hongyi Wang, Eric Xing, Kangwook Lee, Dimitris Papailiopoulos
Machine Learning Faculty Publications
Large neural networks can be pruned to a small fraction of their original size, with little loss in accuracy, by following a time-consuming “train, prune, re-train” approach. Frankle & Carbin [9] conjecture that we can avoid this by training lottery tickets, i.e., special sparse subnetworks found at initialization, that can be trained to high accuracy. However, a subsequent line of work [11, 41] presents concrete evidence that current algorithms for finding trainable networks at initialization, fail simple baseline comparisons, e.g., against training random sparse subnetworks. Finding lottery tickets that train to better accuracy compared to simple baselines remains an open …
Asdot: Any-Shot Data-To-Text Generation With Pretrained Language Models, Jiannan Xiang, Zhengzhong Liu, Yucheng Zhou, Eric P. Xing, Zhiting Hu
Asdot: Any-Shot Data-To-Text Generation With Pretrained Language Models, Jiannan Xiang, Zhengzhong Liu, Yucheng Zhou, Eric P. Xing, Zhiting Hu
Machine Learning Faculty Publications
Data-to-text generation is challenging due to the great variety of the input data in terms of domains (e.g., finance vs sports) or schemata (e.g., diverse predicates). Recent end-to-end neural methods thus require substantial training examples to learn to disambiguate and describe the data. Yet, real-world data-to-text problems often suffer from various data-scarce issues: one may have access to only a handful of or no training examples, and/or have to rely on examples in a different domain or schema. To fill this gap, we propose Any-Shot Data-to-Text (ASDOT), a new approach flexibly applicable to diverse settings by making efficient use of …
Impact Of Digital Twins And Metaverse On Cities: History, Current Situation, And Application Perspectives, Zhihan Lv, Wen Long Shang, Mohsen Guizani
Impact Of Digital Twins And Metaverse On Cities: History, Current Situation, And Application Perspectives, Zhihan Lv, Wen Long Shang, Mohsen Guizani
Machine Learning Faculty Publications
To promote the expansion and adoption of Digital Twins (DTs) in Smart Cities (SCs), a detailed review of the impact of DTs and digitalization on cities is made to assess the progression of cities and standardization of their management mode. Combined with the technical elements of DTs, the coupling effect of DTs technology and urban construction and the internal logic of DTs technology embedded in urban construction are discussed. Relevant literature covering the full range of DTs technologies and their applications is collected, evaluated, and collated, relevant studies are concatenated, and relevant accepted conclusions are summarized by modules. First, the …
Resel: N-Ary Relation Extraction From Scientific Text And Tables By Learning To Retrieve And Select, Yuchen Zhuang, Yinghao Li, Jerry Junyang Cheung, Yue Yu, Yingjun Mou, Xiang Chen, Le Song, Chao Zhang
Resel: N-Ary Relation Extraction From Scientific Text And Tables By Learning To Retrieve And Select, Yuchen Zhuang, Yinghao Li, Jerry Junyang Cheung, Yue Yu, Yingjun Mou, Xiang Chen, Le Song, Chao Zhang
Machine Learning Faculty Publications
We study the problem of extracting N-ary relation tuples from scientific articles. This task is challenging because the target knowledge tuples can reside in multiple parts and modalities of the document. Our proposed method RESEL decomposes this task into a two-stage procedure that first retrieves the most relevant paragraph/table and then selects the target entity from the retrieved component. For the high-level retrieval stage, RESEL designs a simple and effective feature set, which captures multilevel lexical and semantic similarities between the query and components. For the low-level selection stage, RESEL designs a cross-modal entity correlation graph along with a multi-view …
Amp: Automatically Finding Model Parallel Strategies With Heterogeneity Awareness, Dacheng Li, Hongyi Wang, Eric Xing, Hao Zhang
Amp: Automatically Finding Model Parallel Strategies With Heterogeneity Awareness, Dacheng Li, Hongyi Wang, Eric Xing, Hao Zhang
Machine Learning Faculty Publications
Scaling up model sizes can lead to fundamentally new capabilities in many machine learning (ML) tasks. However, training big models requires strong distributed system expertise to carefully design model-parallel execution strategies that suit the model architectures and cluster setups. In this paper, we develop AMP, a framework that automatically derives such strategies. AMP identifies a valid space of model parallelism strategies and efficiently searches the space for high-performed strategies, by leveraging a cost model designed to capture the heterogeneity of the model and cluster specifications. Unlike existing methods, AMP is specifically tailored to support complex models composed of uneven layers …
Efficient (Soft) Q-Learning For Text Generation With Limited Good Data, Han Guo, Bowen Tan, Zhengzhong Liu, Eric P. Xing, Zhiting Hu
Efficient (Soft) Q-Learning For Text Generation With Limited Good Data, Han Guo, Bowen Tan, Zhengzhong Liu, Eric P. Xing, Zhiting Hu
Machine Learning Faculty Publications
Maximum likelihood estimation (MLE) is the predominant algorithm for training text generation models. This paradigm relies on direct supervision examples, which is not applicable to many emerging applications, such as generating adversarial attacks or generating prompts to control language models. Reinforcement learning (RL) on the other hand offers a more flexible solution by allowing users to plug in arbitrary task metrics as reward. Yet previous RL algorithms for text generation, such as policy gradient (on-policy RL) and Q-learning (off-policy RL), are often notoriously inefficient or unstable to train due to the large sequence space and the sparse reward received only …
On Pac Learning Halfspaces In Non-Interactive Local Privacy Model With Public Unlabeled Data, Jinyan Su, Jinhui Xu, Di Wang
On Pac Learning Halfspaces In Non-Interactive Local Privacy Model With Public Unlabeled Data, Jinyan Su, Jinhui Xu, Di Wang
Machine Learning Faculty Publications
In this paper, we study the problem of PAC learning halfspaces in the non-interactive local differential privacy model (NLDP). To breach the barrier of exponential sample complexity, previous results studied a relaxed setting where the server has access to some additional public but unlabeled data. We continue in this direction. Specifically, we consider the problem under the standard setting instead of the large margin setting studied before. Under different mild assumptions on the underlying data distribution, we propose two approaches that are based on the Massart noise model and self-supervised learning and show that it is possible to achieve sample …
Unpaired Image-To-Image Translation With Density Changing Regularization, Shaoan Xie, Qirong Ho, Kun Zhang
Unpaired Image-To-Image Translation With Density Changing Regularization, Shaoan Xie, Qirong Ho, Kun Zhang
Machine Learning Faculty Publications
Unpaired image-to-image translation aims to translate an input image to another domain such that the output image looks like an image from another domain while important semantic information are preserved. Inferring the optimal mapping with unpaired data is impossible without making any assumptions. In this paper, we make a density changing assumption where image patches of high probability density should be mapped to patches of high probability density in another domain. Then we propose an efficient way to enforce this assumption: we train the flows as density estimators and penalize the variance of density changes. Despite its simplicity, our method …
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points Without Gradients, Hualin Zhang, Huan Xiong, Bin Gu
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points Without Gradients, Hualin Zhang, Huan Xiong, Bin Gu
Machine Learning Faculty Publications
We consider escaping saddle points of nonconvex problems where only the function evaluations can be accessed. Although a variety of works have been proposed, the majority of them require either second or first-order information, and only a few of them have exploited zeroth-order methods, particularly the technique of negative curvature finding with zeroth-order methods which has been proven to be the most efficient method for escaping saddle points. To fill this gap, in this paper, we propose two zeroth-order negative curvature finding frameworks that can replace Hessian-vector product computations without increasing the iteration complexity. We apply the proposed frameworks to …
Zeroth-Order Hard-Thresholding: Gradient Error Vs. Expansivity, William De Vazelhes, Hualin Zhang, Huimin Wu, Xiao Tong Yuan, Bin Gu
Zeroth-Order Hard-Thresholding: Gradient Error Vs. Expansivity, William De Vazelhes, Hualin Zhang, Huimin Wu, Xiao Tong Yuan, Bin Gu
Machine Learning Faculty Publications
ℓ0 constrained optimization is prevalent in machine learning, particularly for high-dimensional problems, because it is a fundamental approach to achieve sparse learning. Hard-thresholding gradient descent is a dominant technique to solve this problem. However, first-order gradients of the objective function may be either unavailable or expensive to calculate in a lot of real-world problems, where zeroth-order (ZO) gradients could be a good surrogate. Unfortunately, whether ZO gradients can work with the hard-thresholding operator is still an unsolved problem. To solve this puzzle, in this paper, we focus on the ℓ0 constrained black-box stochastic optimization problems, and propose a new stochastic …
Hyperfast Second-Order Local Solvers For Efficient Statistically Preconditioned Distributed Optimization, Pavel Dvurechensky, Dmitry Kamzolov, Aleksandr Lukashevich, Soomin Lee, Erik Ordentlich, César A. Uribe, Alexander Gasnikov
Hyperfast Second-Order Local Solvers For Efficient Statistically Preconditioned Distributed Optimization, Pavel Dvurechensky, Dmitry Kamzolov, Aleksandr Lukashevich, Soomin Lee, Erik Ordentlich, César A. Uribe, Alexander Gasnikov
Machine Learning Faculty Publications
Statistical preconditioning enables fast methods for distributed large-scale empirical risk minimization problems. In this approach, multiple worker nodes compute gradients in parallel, which are then used by the central node to update the parameter by solving an auxiliary (preconditioned) smaller-scale optimization problem. The recently proposed Statistically Preconditioned Accelerated Gradient (SPAG) method [1] has complexity bounds superior to other such algorithms but requires an exact solution for computationally intensive auxiliary optimization problems at every iteration. In this paper, we propose an Inexact SPAG (InSPAG) and explicitly characterize the accuracy by which the corresponding auxiliary subproblem needs to be solved to guarantee …
Lemurs Optimizer: A New Metaheuristic Algorithm For Global Optimization, Ammar Kamal Abasi, Sharif Naser Makhadmeh, Mohammed Azmi Al-Betar, Osama Ahmad Alomari, Mohammed A. Awadallah, Zaid Abdi Alkareem Alyasseri, Iyad Abu Doush, Ashraf Elnagar, Eman H. Alkhammash, Myriam Hadjouni
Lemurs Optimizer: A New Metaheuristic Algorithm For Global Optimization, Ammar Kamal Abasi, Sharif Naser Makhadmeh, Mohammed Azmi Al-Betar, Osama Ahmad Alomari, Mohammed A. Awadallah, Zaid Abdi Alkareem Alyasseri, Iyad Abu Doush, Ashraf Elnagar, Eman H. Alkhammash, Myriam Hadjouni
Machine Learning Faculty Publications
The Lemur Optimizer (LO) is a novel nature-inspired algorithm we propose in this paper. This algorithm’s primary inspirations are based on two pillars of lemur behavior: leap up and dance hub. These two principles are mathematically modeled in the optimization context to handle local search, exploitation, and exploration search concepts. The LO is first benchmarked on twenty-three standard optimization functions. Additionally, the LO is used to solve three real-world problems to evaluate its performance and effectiveness. In this direction, LO is compared to six well-known algorithms: Salp Swarm Algorithm (SSA), Artificial Bee Colony (ABC), Sine Cosine Algorithm (SCA), Bat Algorithm …
Stochastic Trajectory Prediction Via Motion Indeterminacy Diffusion, Tianpei Gu, Guangyi Chen, Junlong Li, Chunze Lin, Yongming Rao, Jie Zhou, Jiwen Lu
Stochastic Trajectory Prediction Via Motion Indeterminacy Diffusion, Tianpei Gu, Guangyi Chen, Junlong Li, Chunze Lin, Yongming Rao, Jie Zhou, Jiwen Lu
Machine Learning Faculty Publications
Human behavior has the nature of indeterminacy, which requires the pedestrian trajectory prediction system to model the multi-modality of future motion states. Unlike existing stochastic trajectory prediction methods which usually use a latent variable to represent multi-modality, we explicitly simulate the process of human motion variation from indeterminate to determinate. In this paper, we present a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID), in which we progressively discard indeterminacy from all the walkable areas until reaching the desired trajectory. This process is learned with a parameterized Markov chain conditioned by …
Maximum Spatial Perturbation Consistency For Unpaired Image-To-Image Translation, Yanwu Xu, Shaoan Xie, Wenhao Wu, Kun Zhang, Mingming Gong, Kayhan Batmanghelich
Maximum Spatial Perturbation Consistency For Unpaired Image-To-Image Translation, Yanwu Xu, Shaoan Xie, Wenhao Wu, Kun Zhang, Mingming Gong, Kayhan Batmanghelich
Machine Learning Faculty Publications
Unpaired image-to-image translation (I2I) is an ill-posed problem, as an infinite number of translation functions can map the source domain distribution to the target distribution. Therefore, much effort has been put into designing suitable constraints, e.g., cycle consistency (CycleGAN), geometry consistency (GCGAN), and contrastive learning-based constraints (CUTGAN), that help better pose the problem. However, these well-known constraints have limitations: (1) they are either too restrictive or too weak for specific I2I tasks; (2) these methods result in content distortion when there is a significant spatial variation between the source and target domains. This paper proposes a universal regularization technique called …
Learning Hierarchical Metrical Structure Beyond Measures, Junyan Jiang, Daniel Chin, Yixiao Zhang, Gus Xia
Learning Hierarchical Metrical Structure Beyond Measures, Junyan Jiang, Daniel Chin, Yixiao Zhang, Gus Xia
Machine Learning Faculty Publications
Music contains hierarchical structures beyond beats and measures. While hierarchical structure annotations are helpful for music information retrieval and computer musicology, such annotations are scarce in current digital music databases. In this paper, we explore a data-driven approach to automatically extract hierarchical metrical structures from scores. We propose a new model with a Temporal Convolutional Network-Conditional Random Field (TCN-CRF) architecture. Given a symbolic music score, our model takes in an arbitrary number of voices in a beat-quantized form, and predicts a 4-level hierarchical metrical structure from downbeat-level to section-level. We also annotate a dataset using RWC-POP MIDI files to facilitate …
Artificial Intelligence-Driven Design Of Fuel Mixtures, Nursulu Kuzhagaliyeva, Samuel Horváth, John Williams, Andre Nicolle, S. Mani Sarathy
Artificial Intelligence-Driven Design Of Fuel Mixtures, Nursulu Kuzhagaliyeva, Samuel Horváth, John Williams, Andre Nicolle, S. Mani Sarathy
Machine Learning Faculty Publications
High-performance fuel design is imperative to achieve cleaner burning and high-efficiency engine systems. We introduce a data-driven artificial intelligence (AI) framework to design liquid fuels exhibiting tailor-made properties for combustion engine applications to improve efficiency and lower carbon emissions. The fuel design approach is a constrained optimization task integrating two parts: (i) a deep learning (DL) model to predict the properties of pure components and mixtures and (ii) search algorithms to efficiently navigate in the chemical space. Our approach presents the mixture-hidden vector as a linear combination of each single component’s vectors in each blend and incorporates it into the …
Domain Adversarial Training On Conditional Variational Auto-Encoder For Controllable Music Generation, Jingwei Zhao, Gus Xia, Ye Wang
Domain Adversarial Training On Conditional Variational Auto-Encoder For Controllable Music Generation, Jingwei Zhao, Gus Xia, Ye Wang
Machine Learning Faculty Publications
The variational auto-encoder has become a leading framework for symbolic music generation, and a popular research direction is to study how to effectively control the generation process. A straightforward way is to control a model using different conditions during inference. However, in music practice, conditions are usually sequential (rather than simple categorical labels), involving rich information that overlaps with the learned representation. Consequently, the decoder gets confused about whether to “listen to” the latent representation or the condition, and sometimes just ignores the condition. To solve this problem, we leverage domain adversarial training to disentangle the representation from condition cues …
Beat Transformer: Demixed Beat And Downbeat Tracking With Dilated Self-Attention, Jingwei Zhao, Gus Xia, Ye Wang
Beat Transformer: Demixed Beat And Downbeat Tracking With Dilated Self-Attention, Jingwei Zhao, Gus Xia, Ye Wang
Machine Learning Faculty Publications
We propose Beat Transformer, a novel Transformer encoder architecture for joint beat and downbeat tracking. Different from previous models that track beats solely based on the spectrogram of an audio mixture, our model deals with demixed spectrograms with multiple instrument channels. This is inspired by the fact that humans perceive metrical structures from richer musical contexts, such as chord progression and instrumentation. To this end, we develop a Transformer model with both time-wise attention and instrument-wise attention to capture deep-buried metrical cues. Moreover, our model adopts a novel dilated self-attention mechanism, which achieves powerful hierarchical modelling with only linear complexity. …
Decentralized Personalized Federated Learning: Lower Bounds And Optimal Algorithm For All Personalization Modes, Abdurakhmon Sadiev, Ekaterina Borodich, Aleksandr Beznosikov, Darina Dvinskikh, Saveliy Chezhegov, Rachael Tappenden, Martin Takac, Alexander Gasnikov
Decentralized Personalized Federated Learning: Lower Bounds And Optimal Algorithm For All Personalization Modes, Abdurakhmon Sadiev, Ekaterina Borodich, Aleksandr Beznosikov, Darina Dvinskikh, Saveliy Chezhegov, Rachael Tappenden, Martin Takac, Alexander Gasnikov
Machine Learning Faculty Publications
This paper considers the problem of decentralized, personalized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the local model and its average, is often added to the objective function. However, in a decentralized setting this penalty is expensive in terms of communication costs, so here, a different penalty — one that is built to respect the structure of the underlying computational network — is used instead. We present lower bounds on the communication and local computation costs for this problem formulation and we also present provably optimal methods for decentralized personalized federated learning. Numerical …
Accomontage2: A Complete Harmonization And Accompaniment Arrangement System, Li Yi, Haochen Hu, Jingwei Zhao, Gus Xia
Accomontage2: A Complete Harmonization And Accompaniment Arrangement System, Li Yi, Haochen Hu, Jingwei Zhao, Gus Xia
Machine Learning Faculty Publications
We propose AccoMontage2, a system capable of doing full-length song harmonization and accompaniment arrangement based on a lead melody. Following AccoMontage, this study focuses on generating piano arrangements for popular/folk songs and it carries on the generalized template-based retrieval method. The novelties of this study are twofold. First, we invent a harmonization module (which AccoMontage does not have). This module generates structured and coherent full-length chord progression by optimizing and balancing three loss terms: a micro-level loss for note-wise dissonance, a meso-level loss for phrase-template matching, and a macro-level loss for full piece coherency. Second, we develop a graphical user …
Negational Symmetry Of Quantum Neural Networks For Binary Pattern Classification, Nanqing Dong, Michael Kampffmeyer, Irina Voiculescu, Eric P. Xing
Negational Symmetry Of Quantum Neural Networks For Binary Pattern Classification, Nanqing Dong, Michael Kampffmeyer, Irina Voiculescu, Eric P. Xing
Machine Learning Faculty Publications
Although quantum neural networks (QNNs) have shown promising results in solving simple machine learning tasks recently, the behavior of QNNs in binary pattern classification is still underexplored. In this work, we find that QNNs have an Achilles’ heel in binary pattern classification. To illustrate this point, we provide a theoretical insight into the properties of QNNs by presenting and analyzing a new form of symmetry embedded in a family of QNNs with full entanglement, which we term negational symmetry. Due to negational symmetry, QNNs can not differentiate between a quantum binary signal and its negational counterpart. We empirically evaluate the …
Truncated Matrix Power Iteration For Differentiable Dag Learning, Zhen Zhang, Ignavier Ng, Dong Gong, Yuhang Liu, Ehsan M. Abbasnejad, Mingming Gong, Kun Zhang, Javen Qinfeng Shi
Truncated Matrix Power Iteration For Differentiable Dag Learning, Zhen Zhang, Ignavier Ng, Dong Gong, Yuhang Liu, Ehsan M. Abbasnejad, Mingming Gong, Kun Zhang, Javen Qinfeng Shi
Machine Learning Faculty Publications
Recovering underlying Directed Acyclic Graph structures (DAG) from observational data is highly challenging due to the combinatorial nature of the DAG-constrained optimization problem. Recently, DAG learning has been cast as a continuous optimization problem by characterizing the DAG constraint as a smooth equality one, generally based on polynomials over adjacency matrices. Existing methods place very small coefficients on high-order polynomial terms for stabilization, since they argue that large coefficients on the higher-order terms are harmful due to numeric exploding. On the contrary, we discover that large coefficients on higher-order terms are beneficial for DAG learning, when the spectral radiuses of …
Exploiting Higher-Order Derivatives In Convex Optimization Methods, Dmitry Kamzolov, Alexander Gasnikov, Pavel Dvurechensky, Artem Agafonov, Martin Takac
Exploiting Higher-Order Derivatives In Convex Optimization Methods, Dmitry Kamzolov, Alexander Gasnikov, Pavel Dvurechensky, Artem Agafonov, Martin Takac
Machine Learning Faculty Publications
Exploiting higher-order derivatives in convex optimization is known at least since 1970’s. In each iteration higher-order (also called tensor) methods minimize a regularized Taylor expansion of the objective function, which leads to faster convergence rates if the corresponding higher-order derivative is Lipschitz-continuous. Recently a series of lower iteration complexity bounds for such methods were proved, and a gap between upper an lower complexity bounds was revealed. Moreover, it was shown that such methods can be implementable since the appropriately regularized Taylor expansion of a convex function is also convex and, thus, can be minimized in polynomial time. Only very recently …
Sr-Dcsk Cooperative Communication System With Code Index Modulation: A New Design For 6g New Radios, Yi Fang, Wang Chen, Pingping Chen, Yiwei Tao, Mohsen Guizani
Sr-Dcsk Cooperative Communication System With Code Index Modulation: A New Design For 6g New Radios, Yi Fang, Wang Chen, Pingping Chen, Yiwei Tao, Mohsen Guizani
Machine Learning Faculty Publications
This paper proposes a high-throughput short reference differential chaos shift keying cooperative communication system with the aid of code index modulation, referred to as CIM-SR-DCSK-CC system. In the proposed CIM-SR-DCSK-CC system, the source transmits information bits to both the relay and destination in the first time slot, while the relay not only forwards the source information bits but also sends new information bits to the destination in the second time slot. To be specific, the relay employs an N-order Walsh code to carry additional log2N information bits, which are superimposed onto the SR-DCSK signal carrying the decoded source information bits. …
Interpreting Song Lyrics With An Audio-Informed Pre-Trained Language Model, Yixiao Zhang, Junyan Jiang, Gus Xia, Simon Dixon
Interpreting Song Lyrics With An Audio-Informed Pre-Trained Language Model, Yixiao Zhang, Junyan Jiang, Gus Xia, Simon Dixon
Machine Learning Faculty Publications
Lyric interpretations can help people understand songs and their lyrics quickly, and can also make it easier to manage, retrieve and discover songs efficiently from the growing mass of music archives. In this paper we propose BART-fusion, a novel model for generating lyric interpretations from lyrics and music audio that combines a large-scale pre-trained language model with an audio encoder. We employ a cross-modal attention module to incorporate the audio representation into the lyrics representation to help the pre-trained language model understand the song from an audio perspective, while preserving the language model’s original generative performance. We also release the …