Delving Into Multimodal Prompting For Fine-Grained Visual Classification,
2024
Singapore Management University
Delving Into Multimodal Prompting For Fine-Grained Visual Classification, Xin Jiang, Hao Tang, Junyao Gao, Xiaoyu Du, Shengfeng He, Zechao Li
Research Collection School Of Computing and Information Systems
Fine-grained visual classification (FGVC) involves categorizing fine subdivisions within a broader category, which poses challenges due to subtle inter-class discrepancies and large intra-class variations. However, prevailing approaches primarily focus on uni-modal visual concepts. Recent advancements in pre-trained vision-language models have demonstrated remarkable performance in various high-level vision tasks, yet the applicability of such models to FGVC tasks remains uncertain. In this paper, we aim to fully exploit the capabilities of cross-modal description to tackle FGVC tasks and propose a novel multimodal prompting solution, denoted as MP-FGVC, based on the contrastive language-image pertaining (CLIP) model. Our MP-FGVC comprises a multimodal prompts …
Leveraging Llms And Generative Models For Interactive Known-Item Video Search,
2024
Singapore Management University
Leveraging Llms And Generative Models For Interactive Known-Item Video Search, Zhixin Ma, Jiaxin Wu, Chong-Wah Ngo
Research Collection School Of Computing and Information Systems
While embedding techniques such as CLIP have considerably boosted search performance, user strategies in interactive video search still largely operate on a trial-and-error basis. Users are often required to manually adjust their queries and carefully inspect the search results, which greatly rely on the users’ capability and proficiency. Recent advancements in large language models (LLMs) and generative models offer promising avenues for enhancing interactivity in video retrieval and reducing the personal bias in query interpretation, particularly in the known-item search. Specifically, LLMs can expand and diversify the semantics of the queries while avoiding grammar mistakes or the language barrier. In …
Mitigating Fine-Grained Hallucination By Fine-Tuning Large Vision-Language Models With Caption Rewrites,
2024
Singapore Management University
Mitigating Fine-Grained Hallucination By Fine-Tuning Large Vision-Language Models With Caption Rewrites, Lei Wang, Jiabang He, Shenshen Li, Ning Liu, Ee-Peng Lim
Research Collection School Of Computing and Information Systems
Large language models (LLMs) have shown remarkable performance in natural language processing (NLP) tasks. To comprehend and execute diverse human instructions over image data, instruction-tuned large vision-language models (LVLMs) have been introduced. However, LVLMs may suffer from different types of object hallucinations. Nevertheless, LVLMs are evaluated for coarse-grained object hallucinations only (i.e., generated objects non-existent in the input image). The fine-grained object attributes and behaviors non-existent in the image may still be generated but not measured by the current evaluation methods. In this paper, we thus focus on reducing fine-grained hallucinations of LVLMs. We propose ReCaption, a framework that consists …
Earnhft: Efficient Hierarchical Reinforcement Learning For High Frequency Trading,
2024
Singapore Management University
Earnhft: Efficient Hierarchical Reinforcement Learning For High Frequency Trading, Molei Qin, Shuo Sun, Wentao Zhang, Haochong Xia, Xinrun Wang, Bo An
Research Collection School Of Computing and Information Systems
High-frequency trading (HFT) is using computer algorithms to make trading decisions in short time scales (e.g., second-level), which is widely used in the Cryptocurrency (Crypto) market, (e.g., Bitcoin). Reinforcement learning (RL) in financial research has shown stellar performance on many quantitative trading tasks. However, most methods focus on low-frequency trading, e.g., day-level, which cannot be directly applied to HFT because of two challenges. First, RL for HFT involves dealing with extremely long trajectories (e.g., 2.4 million steps per month), which is hard to optimize and evaluate. Second, the dramatic price fluctuations and market trend changes of Crypto make existing algorithms …
Transition-Informed Reinforcement Learning For Large-Scale Stackelberg Mean-Field Games.,
2024
Singapore Management University
Transition-Informed Reinforcement Learning For Large-Scale Stackelberg Mean-Field Games., Pengdeng Li, Runsheng Yu, Xinrun Wang, Bo An
Research Collection School Of Computing and Information Systems
Many real-world scenarios including fleet management and Ad auctions can be modeled as Stackelberg mean-field games (SMFGs) where a leader aims to incentivize a large number of homogeneous self-interested followers to maximize her utility. Existing works focus on cases with a small number of heterogeneous followers, e.g., 5-10, and suffer from scalability issue when the number of followers increases. There are three major challenges in solving large-scale SMFGs: i) classical methods based on solving differential equations fail as they require exact dynamics parameters, ii) learning by interacting with environment is data-inefficient, and iii) complex interaction between the leader and followers …
Unsupervised Training Sequence Design: Efficient And Generalizable Agent Training,
2024
Singapore Management University
Unsupervised Training Sequence Design: Efficient And Generalizable Agent Training, Wenjun Li, Pradeep Varakantham
Research Collection School Of Computing and Information Systems
To train generalizable Reinforcement Learning (RL) agents, researchers recently proposed the Unsupervised Environment Design (UED) framework, in which a teacher agent creates a very large number of training environments and a student agent trains on the experiences in these environments to be robust against unseen testing scenarios. For example, to train a student to master the “stepping over stumps” task, the teacher will create numerous training environments with varying stump heights and shapes. In this paper, we argue that UED neglects training efficiency and its need for very large number of environments (henceforth referred to as infinite horizon training) makes …
Out-Of-Distribution Detection In Long-Tailed Recognition With Calibrated Outlier Class Learning,
2024
Singapore Management University
Out-Of-Distribution Detection In Long-Tailed Recognition With Calibrated Outlier Class Learning, Wenjun Miao, Guansong Pang, Xiao Bai, Tianqi Li, Jin Zheng
Research Collection School Of Computing and Information Systems
Existing out-of-distribution (OOD) methods have shown great success on balanced datasets but become ineffective in long-tailed recognition (LTR) scenarios where 1) OOD samples are often wrongly classified into head classes and/or 2) tail-class samples are treated as OOD samples. To address these issues, current studies fit a prior distribution of auxiliary/pseudo OOD data to the long-tailed in-distribution (ID) data. However, it is difficult to obtain such an accurate prior distribution given the unknowingness of real OOD samples and heavy class imbalance in LTR. A straightforward solution to avoid the requirement of this prior is to learn an outlier class to …
Vadclip: Adapting Vision-Language Models For Weakly Supervised Video Anomaly Detection,
2024
Singapore Management University
Vadclip: Adapting Vision-Language Models For Weakly Supervised Video Anomaly Detection, Peng Wu, Xuerong Zhou, Guansong Pang, Lingru Zhou, Qingsen Yan, Peng Wang, Yanning Zhang
Research Collection School Of Computing and Information Systems
The recent contrastive language-image pre-training (CLIP) model has shown great success in a wide range of image-level tasks, revealing remarkable ability for learning powerful visual representations with rich semantics. An open and worthwhile problem is efficiently adapting such a strong model to the video domain and designing a robust video anomaly detector. In this work, we propose VadCLIP, a new paradigm for weakly supervised video anomaly detection (WSVAD) by leveraging the frozen CLIP model directly without any pre-training and fine-tuning process. Unlike current works that directly feed extracted features into the weakly supervised classifier for frame-level binary classification, VadCLIP makes …
Does Chatgpt Know Calculus?,
2024
St. John Fisher University
Does Chatgpt Know Calculus?, Kris H. Green
Journal of Humanistic Mathematics
Academics and educators across the world are grappling with how OpenAI’s new software, ChatGPT, will impact teaching and learning. This essay explores ChatGPT’s response to a typical calculus problem as a way of illustrating its functionality and limitations.
Dynamic Prognosis Prediction For Patients On Dapt After Drug-Eluting Stent Implantation: Model Development And Validation,
2024
University of Texas Health Science Center at Houston
Dynamic Prognosis Prediction For Patients On Dapt After Drug-Eluting Stent Implantation: Model Development And Validation, Fang Li, Laila Rasmy, Yang Xiang, Jingna Feng, Ahmed Abdelhameed, Xinyue Hu, Zenan Sun, David Aguilar, Abhijeet Dhoble, Jingcheng Du, Qing Wang, Shuteng Niu, Yifang Dang, Xinyuan Zhang, Ziqian Xie, Yi Nian, Jianping He, Yujia Zhou, Jianfu Li, Mattia Prosperi, Jiang Bian, Degui Zhi, Cui Tao
School of Medicine Faculty Publications
BACKGROUND: The rapid evolution of artificial intelligence (AI) in conjunction with recent updates in dual antiplatelet therapy (DAPT) management guidelines emphasizes the necessity for innovative models to predict ischemic or bleeding events after drug-eluting stent implantation. Leveraging AI for dynamic prediction has the potential to revolutionize risk stratification and provide personalized decision support for DAPT management. METHODS AND RESULTS: We developed and validated a new AI-based pipeline using retrospective data of drug-eluting stent-treated patients, sourced from the Cerner Health Facts data set (n=98 236) and Optum’s de-identified Clinformatics Data Mart Database (n=9978). The 36 months following drug-eluting stent implantation were …
Promises And Risks Of Applying Ai Medical Imaging To Early Detection Of Cancers, And Regulation For Ai Medical Imaging,
2024
The Johns Hopkins University
Promises And Risks Of Applying Ai Medical Imaging To Early Detection Of Cancers, And Regulation For Ai Medical Imaging, Yiyao Zhang
The Journal of Purdue Undergraduate Research
No abstract provided.
A Survey On Training Challenges In Generative Adversarial Networks For Biomedical Image Analysis,
2024
Department of Computer Science, Munster Technological University (MTU), Cork, Ireland
A Survey On Training Challenges In Generative Adversarial Networks For Biomedical Image Analysis, Muhammad Muneeb Saad, Ruairi O'Reilly, Mubashir Husain Rehmani
Department of Computer Science Publications
In biomedical image analysis, the applicability of deep learning methods is directly impacted by the quantity of image data available. This is due to deep learning models requiring large image datasets to provide high-level performance. Generative Adversarial Networks (GANs) have been widely utilized to address data limitations through the generation of synthetic biomedical images. GANs consist of two models. The generator, a model that learns how to produce synthetic images based on the feedback it receives. The discriminator, a model that classifies an image as synthetic or real and provides feedback to the generator. Throughout the training process, a GAN …
Natural Language Processing Analysis Of Online Reviews For Small Business: Extracting Insight From Small Corpora,
2024
Air Force Institute of Technology
Natural Language Processing Analysis Of Online Reviews For Small Business: Extracting Insight From Small Corpora, Benjamin J. Mccloskey, Phillip M. Lacasse, Bruce A. Cox
Faculty Publications
Receiving and acting on customer input is essential to sustaining and growing any service organization, particularly a small family business whose livelihood depends on strong relationships with its customers. The competitive advantage offered by advanced analytical approaches for supporting decisions is not trivial, and enterprises across virtually all domains of society are investing heavily in this emerging discipline. Natural Language Processing (NLP) is a subset of computer science that employs computational approaches to analyze human language; it is effective at extracting insight from text data but frequently requires large corpora to train its models, in the scale of thousands or …
De Novo Drug Design Using Transformer-Based Machine Translation And Reinforcement Learning Of An Adaptive Monte Carlo Tree Search,
2024
Chapman University
De Novo Drug Design Using Transformer-Based Machine Translation And Reinforcement Learning Of An Adaptive Monte Carlo Tree Search, Dony Ang, Cyril Rakovski, Hagop S. Atamian
Biology, Chemistry, and Environmental Sciences Faculty Articles and Research
The discovery of novel therapeutic compounds through de novo drug design represents a critical challenge in the field of pharmaceutical research. Traditional drug discovery approaches are often resource intensive and time consuming, leading researchers to explore innovative methods that harness the power of deep learning and reinforcement learning techniques. Here, we introduce a novel drug design approach called drugAI that leverages the Encoder–Decoder Transformer architecture in tandem with Reinforcement Learning via a Monte Carlo Tree Search (RL-MCTS) to expedite the process of drug discovery while ensuring the production of valid small molecules with drug-like characteristics and strong binding affinities towards …
Anthropomorphism And Human-Robot Interaction,
2024
Montclair State University
Anthropomorphism And Human-Robot Interaction, Rae Yule Kim
Department of Economics Faculty Scholarship and Creative Works
Exploring how human appreciation for and interactions with robots are influenced by anthropomorphic features.
Scene Graph Generation: A Comprehensive Survey,
2024
Edith Cowan University
Scene Graph Generation: A Comprehensive Survey, Hongsheng Li, Guangming Zhu, Liang Zhang, Youliang Jiang, Yixuan Dang, Haoran Hou, Peiyi Shen, Xia Zhao, Syed A. A. Shah, Mohammed Bennamoun
Research outputs 2022 to 2026
Deep learning techniques have led to remarkable breakthroughs in the field of object detection and have spawned a lot of scene-understanding tasks in recent years. Scene graph has been the focus of research because of its powerful semantic representation and applications to scene understanding. Scene Graph Generation (SGG) refers to the task of automatically mapping an image or a video into a semantic structural scene graph, which requires the correct labeling of detected objects and their relationships. In this paper, a comprehensive survey of recent achievements is provided. This survey attempts to connect and systematize the existing visual relationship detection …
Spatio-Temporal Association Rule Mining Of Traffic Congestion In A Large-Scale Road Network Based On
Trajectory Data,
2024
School of Transportation and Logistics, Southwest Jiaotong University, Chengdu 611756, China
Spatio-Temporal Association Rule Mining Of Traffic Congestion In A Large-Scale Road Network Based On Trajectory Data, Qifan Zhou, Haixu Liu, Zhipeng Dong, Yin Xu
Journal of System Simulation
Abstract: A K neighbor-RElim (KNR) algorithm and a sequential KNbr-RElim (SKNR) algorithm are proposed to mine traffic congestion association rules and congestion propagation spatio-temporal association rules by vehicle trajectory data in a large-scale road network. The KNR algorithm extends the spatial topology constraint based on the RElim algorithm. The KNR can be used to mine the road links prone to congestion from the large-scale trajectory dataset in a large-scale road network and quantify the strength of association for congested road links. The SKNR algorithm expands the time dimension in the form of sliding window and can be applied for mining …
Modeling And Analysing Of Complex Combat Systems Based On Symbiosis Theory,
2024
College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China
Modeling And Analysing Of Complex Combat Systems Based On Symbiosis Theory, Xiangrui Tian, Jie Ying, Rui Yao, Xiaodong Wan
Journal of System Simulation
Abstract: As the combat systems develop towards clustering, coordination, unmanned, and intelligent direction, traditional combat system modeling methods are unable to reflect the complexity and intelligence of the combat systems. By drawing on symbiosis theory, this paper models and analyzes complex combat systems, and decomposes the complex combat system into various subsystems according to combat missions. The combat units, interaction modes, and combat environments in the subsystems are analyzed. The paper builds mathematical models to capture the collaborative interaction relationships between combat units, finally constructing a symbiotic model of complex combat systems. By the symbiosis principles and methods, quantitative analysis …
Action Recognition Model Of Directed Attention Based On Cosine Similarity,
2024
Command & Control Engineering College, Army Engineering University of PLA, Nanjing 210007, China
Action Recognition Model Of Directed Attention Based On Cosine Similarity, Chen Li, Ming He, Chen Dong, Wei Li
Journal of System Simulation
Abstract: Aiming at the lack of directionality of traditional dot product attention, this paper proposes a directed attention model (DAM) based on cosine similarity. To effectively represent the direction relationship between the spatial and temporal features of video frames, the paper defines the relationship function in the attention mechanism using the cosine similarity theory, which can remove the absolute value of the relationship between features. To reduce the computational burden of the attention mechanism, the operation is decomposed from two dimensions of time and space. The computational complexity is further optimized by combining linear attention operation. The experiment is divided …
Overall Scheme Design And Integration Testing Of Hardware-In-The-Loop Simulation Of Guidance And Control
System,
2024
Unmanned System Research Institute, Northwestern Polytechnical University, Xi'an 710072, China
Overall Scheme Design And Integration Testing Of Hardware-In-The-Loop Simulation Of Guidance And Control System, Xiaofei Chang, Jiayue Jiao, Kang Chen, Wenxing Fu, Jie Yan
Journal of System Simulation
Abstract: Hardware-in-the-loop simulation system is a complex distributed simulation system, and its design and integration directly affect the system performance and construction goals. Based on years of experience, this paper first summarizes the design of the overall scheme and analyzes the performance requirements of real-time, compatibility, scalability, and security. Then, the paper describes the overall scheme of a typical hardware-in-the-loop simulation system, including the functional hierarchy, operation mechanism, and structural composition. Finally, it summarizes the contents and steps of the system integration testing, acceptance testing, and credibility evaluation method.
