Computer Engineering Commons^™

Knowledge-Infused Reinforcement Learning, Kaushik Roy, Manas Gaur, Qi Zhang, Amit Sheth

Publications

Virtual health agents (VHAs) have received considerable attention, but the early focus has been on collecting data, helping patients follow generic health guidelines, and providing reminders for clinical appointments. While presenting the collected data and frequency of visits to the clinician is useful, further context and personalization are needed for a VHA to interpret and understand what the data means in clinical terms. This has made their use in managing health limited. Such understanding enables patient empowerment and self-appraisal – i.e., aiding the patient in interpreting the data to understand the changes in the patient’s health conditions, and self-management – …

Proknow: Process Knowledge For Safety Constrained And Explainable Question Generation For Mental Health Diagnostic Assistance, Kaushik Roy, Manas Gaur, Misagh Soltani, Vipula Rawte, Ashwin Allen, Amit Sheth

Publications

Virtual Mental Health Assistants (VMHAs) are utilized in health care to provide patient services such as counseling and suggestive care. They are not used for patient diagnostic assistance because they cannot adhere to safety constraints and specialized clinical process knowledge (ProKnow) used to obtain clinical diagnoses. In this work, we define ProKnow as an ordered set of information that maps to evidence-based guidelines or categories of conceptual understanding to experts in a domain. We also introduce a new dataset of diagnostic conversations guided by safety constraints and ProKnow that healthcare professionals use (ProKnow-data). We develop a method for natural language …

Defining And Detecting Toxicity On Social Media: Context And Knowledge Are Key, Amit Sheth, Valerie Shalin, Ugur Kursuncu

Publications

As the role of online platforms has become increasingly prominent for communication, toxic behaviors, such as cyberbullying and harassment, have been rampant in the last decade. On the other hand, online toxicity is multi-dimensional and sensitive in nature, which makes its detection challenging. As the impact of exposure to online toxicity can lead to serious implications for individuals and communities, reliable models and algorithms are required for detecting and understanding such communications. In this paper We define toxicity to provide a foundation drawing social theories. Then, we provide an approach that identifies multiple dimensions of toxicity and incorporates explicit knowledge …

A Cdzntese Gamma Spectrometer Trained By Deep Convolutional Neural Network For Radioisotope Identification, Sandeep K. Chaudhuri, Joshua W. Kleppinger, Ritwik Nag, Kaushik Roy, Rojina Panta, Forest Agostinelli, Amit Sheth, Utpal N. Roy, Ralph B. James, Krishna C. Mandal

Publications

We report the implementation of a deep convolutional neural network to train a high-resolution room-temperature CdZnTeSe based gamma ray spectrometer for accurate and precise determination of gamma ray energies for radioisotope identification. The prototype learned spectrometer consists of a NI PCI 5122 fast digitizer connected to a pre-amplifier to recognize spectral features in a sequence of data. We used simulated preamplifier pulses that resemble actual data for various gamma photon energies to train a CNN on the equivalent of 90 seconds worth of data and validated it on 10 seconds worth of simulated data.

Towards Semantic Integration Of Machine Vision Systems To Aid Manufacturing Event Understanding, Kaishu Xia, Clint Saidy, Max Kirkpatrick, Noble Anumbe, Amit Sheth, Ramy Harik

Publications

A manufacturing paradigm shift from conventional control pyramids to decentralized, service-oriented, and cyber-physical systems (CPSs) is taking place in today’s 4th industrial revolution. Generally accepted roles and implementation recipes of cyber systems are expected to be standardized in the future of manufacturing industry. The authors intend to develop a novel CPS-enabled control architecture that accommodates: (1) intelligent information systems involving domain knowledge, empirical model, and simulation; (2) fast and secured industrial communication networks; (3) cognitive automation by rapid signal analytics and machine learning (ML) based feature extraction; (4) interoperability between machine and human. Semantic integration of process indicators is fundamental …

Personalized Digital Phenotype Score, Healthcare Management And Intervention Strategies Using Knowledge Enabled Digital Health Framework For Pediatric Asthma, Utkarshani Jaimini, Amit Sheth

Publications

Asthma is a personalized, and multi-trigger respiratory condition which requires continuous monitoring and management of symptoms and medication adherence. We developed kHealth: Knowledge-enabled Digital Healthcare Framework to monitor and manage the asthma symptoms, medication adherence, lung function, daily activity, sleep quality, indoor, and outdoor environmental triggers of pediatric asthma patients. The kHealth framework collects up to 1852 data points per patient per day. It is practically impossible for the clinicians, parents, and the patient to analyze this vast amount of multimodal data collected from the kHealth framework. In this chapter, we describe the personalized scores, clinically relevant asthma categorization using …

Cyber Social Threats 2021: Ai, Covid-19 Vaccine, Detection And Countering Strategies, Ugur Kursuncu, Jeremy Blackburn, Yelena Mejova, Megan Squire, Amit Sheth

Publications

In recent years, online platforms have been utilized for promoting harmful content and behavior such as extremism, harassment, mis/disinformation, human trafficking, genderbased violence among others affecting our society, often leading to real-world events. Such content and behaviors are inherently complex, making the recognition of these narratives challenging for researchers as well as social media companies. The Cyber Social Threats (CySoc) Workshop 2021 aimed to facilitate a rich forum for researchers and practitioners from both academia and industry in the areas of computing and social science, to discuss novel avenues for research on interdisciplinary aspects of harmful communications on social media, …

Characterization Of Time-Variant And Time-Invariant Assessment Of Suicidality On Reddit Using C-Ssrs, Manas Gaur, Vamsi Aribandi, Amanuel Alambo, Ugur Kursuncu, Krishnaprasad Thirunarayan, Jonathan Beich, Jyotishman Pathak, Amit Sheth

Publications

Suicide is the 10th leading cause of death in the U.S (1999-2019). However, predicting when someone will attempt suicide has been nearly impossible. In the modern world, many individuals suffering from mental illness seek emotional support and advice on well-known and easily-accessible social media platforms such as Reddit. While prior artificial intelligence research has demonstrated the ability to extract valuable information from social media on suicidal thoughts and behaviors, these efforts have not considered both severity and temporality of risk. The insights made possible by access to such data have enormous clinical potential - most dramatically envisioned as a trigger …

Demo Abstract: Porting And Execution Of Anomalies Detection Models On Embedded Systems In Iot, Bharath Sudharsan, Pankesh Patel, Abdul Wahid, Muhammad Yahya, John G. Breslin, Muhammad Intizar Ali

Publications

In the Industry 4.0 era, Microcontrollers (MCUs) based tiny embedded sensor systems have become the sensing paradigm to interact with the physical world. In 2020, 25.6 billion MCUs were shipped, and over 250 billion MCUs are already operating in the wild. Such low-power, low-cost MCUs are being used as the brain to control diverse applications and soon will become the global digital nervous system. In an Industrial IoT setup, such tiny MCU-based embedded systems are equipped with anomaly detection models and mounted on production plant machines for monitoring the machine’s health/condition. These models process the machine’s health data (from temperature, …

Owsnet: Towards Real-Time Offensive Words Spotting Network For Consumer Iot Devices, Bharath Sudharsan, Sweta Malik, Peter Corcoran, Pankesh Patel, John G. Breslin, Muhammad Intizar Ali

Publications

Every modern household owns at least a dozen of IoT devices like smart speakers, video doorbells, smartwatches, where most of them are equipped with a Keyword spotting(KWS) system-based digital voice assistant like Alexa. The state-of-the-art KWS systems require a large number of operations, higher computation, memory resources to show top performance. In this paper, in contrast to existing resource-demanding KWS systems, we propose a light-weight temporal convolution based KWS system named OWSNet, that can comfortably execute on a variety of IoT devices around us and can accurately spot multiple keywords in real-time without disturbing the device's routine functionalities.

When OWSNet …

Cognitive Digital Twins For Smart Manufacturing, Muhammad Intizar Ali, Pankesh Patel, John G. Breslin, Ramy Harik, Amit Sheth

Publications

Smart manufacturing or Industry 4.0, a trend initiated a decade ago, aims to revolutionize traditional manufacturing using technology-driven approaches. Modern digital technologies such as the Industrial Internet of Things (IIoT), Big Data Analytics, Augmented/Virtual Reality, and Artificial Intelligence (AI) are the key enablers of new smart manufacturing approaches. The digital twin is an emerging concept whereby a digital replica can be built of any physical object. Digital twins are becoming mainstream; many organizations have started to rely on digital twins to monitor, analyze, and simulate physical assets and processes. The current use of digital twins for smart manufacturing is largely …

Machine Learning Meets Internet Of Things: From Theory To Practice, Bharath Sudharsan, Pankesh Patel

Publications

Standalone execution of problem-solving Artificial Intelligence (AI) on IoT devices produces a higher level of autonomy and privacy. This is because the sensitive user data collected by the devices need not be transmitted to the cloud for inference. The chipsets used to design IoT devices are resource-constrained due to their limited memory footprint, fewer computation cores, and low clock speeds. These limitations constrain one from deploying and executing complex problem-solving AI (usually an ML model) on IoT devices. Since there is a high potential for building intelligent IoT devices, in this tutorial, we teach researchers and developers; (i) How to …

"When They Say Weed Causes Depression, But It's Your Fav Antidepressant": Knowledge-Aware Attention Framework For Relationship Extraction, Shweta Yadav, Usha Lokala, Raminta Daniulaityte, Krishnaprasad Thirunarayan, Francois Lamy, Amit Sheth

Publications

With the increasing legalization of medical and recreational use of cannabis, more research is needed to understand the association between depression and consumer behavior related to cannabis consumption. Big social media data has potential to provide deeper insights about these associations to public health analysts. In this interdisciplinary study, we demonstrate the value of incorporating domain-specific knowledge in the learning process to identify the relationships between cannabis use and depression. We develop an end-to-end knowledge infused deep learning framework (Gated-K-BERT) that leverages the pre-trained BERT language representation model and domain-specific declarative knowledge source (Drug Abuse Ontology (DAO)) to jointly extract …

Knowledge Infused Policy Gradients For Adaptive Pandemic Control, Kaushik Roy, Qi Zhang, Manas Gaur, Amit P. Sheth

Publications

COVID-19 has impacted nations differently based on their policy implementations. The effective policy requires taking into account public information and adaptability to new knowledge. Epidemiological models built to understand COVID-19 seldom provide the policymaker with the capability for adaptive pandemic control (APC). Among the core challenges to be overcome include (a) inability to handle a high degree of non-homogeneity in different contributing features across the pandemic timeline, (b) lack of an approach that enables adaptive incorporation of public health expert knowledge, and (c) transparent models that enable understanding of the decision-making process in suggesting policy. In this work, we take …

"Is Depression Related To Cannabis?": A Knowledge-Infused Model For Entity And Relation Extraction With Limited Supervision, Kaushik Roy, Usha Lokala, Vedant Khandelwal, Amit P. Sheth

Publications

With strong marketing advocacy of the benefits of cannabis use for improved mental health, cannabis legalization is a priority among legislators. However, preliminary scientific research does not conclusively associate cannabis with improved mental health. In this study, we explore the relationship between depression and consumption of cannabis in a targeted social media corpus involving personal use of cannabis with the intent to derive its potential mental health benefit. We use tweets that contain an association among three categories annotated by domain experts - Reason, Effect, and Addiction. The state-of-the-art Natural Langauge Processing techniques fall short in extracting these relationships between …

Nlp Is Not Enough - Contextualization Of User Input In Chatbots, Nathan Dolbir, Triyasha Dastidar, Kaushik Roy

Publications

AI chatbots have made vast strides in technology improvement in recent years and are already operational in many industries. Advanced Natural Language Processing techniques, based on deep networks, efficiently process user requests to carry out their functions. As chatbots gain traction, their applicability in healthcare is an attractive proposition due to the reduced economic and people costs of an overburdened system. However, healthcare bots require safe and medically accurate information capture, which deep networks aren’t yet capable of due to user text and speech variations. Knowledge in symbolic structures is more suited for accurate reasoning but cannot handle natural language …

Knowledge Infused Policy Gradients With Upper Confidence Bound For Relational Bandits, Kaushik Roy, Qi Zhang, Manas Gaur, Amit Sheth

Publications

Contextual Bandits find important use cases in various real-life scenarios such as online advertising, recommendation systems, healthcare, etc. However, most of the algorithms use at feature vectors to represent context whereas, in the real world, there is a varying number of objects and relations among them to model in the context. For example, in a music recommendation system, the user context contains what music they listen to, which artists create this music, the artist albums, etc. Adding richer relational context representations also introduces a much larger context space making exploration-exploitation harder. To improve the efficiency of exploration-exploitation knowledge about the …

Semantics Of The Black-Box: Can Knowledge Graphs Help Make Deep Learning Systems More Interpretable And Explainable?, Manas Gaur, Keyur Faldu, Amit Sheth

Publications

The recent series of innovations in deep learning (DL) have shown enormous potential to impact individuals and society, both positively and negatively. The DL models utilizing massive computing power and enormous datasets have significantly outperformed prior historical benchmarks on increasingly difficult, well-defined research tasks across technology domains such as computer vision, natural language processing, signal processing, and human-computer interactions. However, the Black-Box nature of DL models and their over-reliance on massive amounts of data condensed into labels and dense representations poses challenges for interpretability and explainability of the system. Furthermore, DLs have not yet been proven in their ability to …

A Meta-Gradient Approach To Learning Cooperative Multi-Agent Communication Topology, Qi Zhang, Dingyang Chen

Publications

In cooperative multi-agent reinforcement learning (MARL), agents often can only partially observe the environment state, and thus communication is crucial to achieving coordination. Communicating agents must simultaneously learn to whom to communicate (i.e., communication topology) and how to interpret the received message for decision-making. Although agents can efficiently learn communication interpretation by end-to-end backpropagation, learning communication topology is much trickier since the binary decisions of whether to communicate impede end-to-end differentiation. As evidenced in our experiments, existing solutions, such as reparameterization tricks and reformulating topology learning as reinforcement learning, often fall short. This paper introduces a meta-learning framework that aims …

Enabling Machine Learning On The Edge Using Sram Conserving Efficient Neural Networks Execution Approach, Bharath Sudharsan, Pankesh Patel, John G. Breslin, Muhammad Intizar Ali

Publications

Edge analytics refers to the application of data analytics and Machine Learning (ML) algorithms on IoT devices. The concept of edge analytics is gaining popularity due to its ability to perform AI-based analytics at the device level, enabling autonomous decision-making, without depending on the cloud. However, the majority of Internet of Things (IoT) devices are embedded systems with a low-cost microcontroller unit (MCU) or a small CPU as its brain, which often are incapable of handling complex ML algorithms.

In this paper, we propose an approach for the ecient execution of already deeply compressed, large neural networks (NNs) on tiny …

Identifying Depressive Symptoms From Tweets: Figurative Language Enabled Multitask Learning Framework, Shweta Yadav, Jainish Chauhan, Joy Prakash Sain, Krishnaprasad Thirunarayan, Amit P. Sheth, Jeremiah Schumm

Publications

Existing studies on using social media for deriving mental health status of users focus on the depression detection task. However, for case management and referral to psychiatrists, healthcare workers require practical and scalable depressive disorder screening and triage system. This study aims to design and evaluate a decision support system (DSS) to reliably determine the depressive triage level by capturing fine-grained depressive symptoms expressed in user tweets through the emulation of Patient Health Questionnaire-9 (PHQ-9) that is routinely used in clinical practice. The reliable detection of depressive symptoms from tweets is challenging because the 280-character limit on tweets incentivizes the …

Medical Knowledge-Enriched Textual Entailment Framework, Shweta Yadav, Vishal Pallagani, Amit P. Sheth

Publications

One of the cardinal tasks in achieving robust medical question answering systems is textual entailment. The existing approaches make use of an ensemble of pre-trained language models or data augmentation, often to clock higher numbers on the validation metrics. However, two major shortcomings impede higher success in identifying entailment: (1) understanding the focus/intent of the question and (2) ability to utilize the real-world background knowledge to capture the context beyond the sentence. In this paper, we present a novel Medical Knowledge-Enriched Textual Entailment framework that allows the model to acquire a semantic and global representation of the input medical text …

Covid-19 In Spain And India: Comparing Policy Implications By Analyzing Epidemiological And Social Media Data, Parth Asawa, Manas Gaur, Kaushik Roy, Amit P. Sheth

Publications

The COVID-19 pandemic has forced public health experts to develop contingent policies to stem the spread of infection, including measures such as partial/complete lockdowns. The effectiveness of these policies has varied with geography, population distribution, and effectiveness in implementation. Consequently, some nations (e.g., Taiwan, Haiti) have been more successful than others (e.g., United States) in curbing the outbreak. A data-driven investigation into effective public health policies of a country would allow public health experts in other nations to decide future courses of action to control the outbreaks of disease and epidemics. We chose Spain and India to present our analysis …

Is It Safe For My Child’S Asthma?, Utkarshani Jaimini, Amit Sheth, Krishnaprasad Thirunarayan, Maninder Kalra, Marco Valtorta

Publications

kHealth-Asthma, a personalised digital healthcare framework is developed to address the above shortcomings by continuous monitoring of the child’s digital phenotype, indoor, and outdoor environmental data. The kHealth-Asthma study has recruited 140 children (ongoing) with an aim to complete recruitment of 150 children. The study period is either 1 month or 3 month depending on the choice of the study participant. kHealth-Asthma collects 29 multi-modal parameters leading to 1852 data points per patient per day (i.e. deployment: 1 month:1852*30=55,560 data points per patient and 3 month:1852*90=166,680 data points per patient). The digital phenotype collected using the kHealth-Asthma generates a Digital …

Multimodal Mental Health Analysis In Social Media, Amir Hossein Yazdavar, Mohammad Saeid Mahdavinejad, Goonmeet Bajaj, William Romine, Amit P. Sheth, Amir Hassan Monadjemi, Krishnaprasad Thirunarayan, John M. Meddar, Annie Myers, Jyotishman Pathak, Pascal Hitzler

Publications

Depression is a major public health concern in the U.S. and globally. While successful early identification and treatment can lead to many positive health and behavioral outcomes, depression, remains undiagnosed, untreated or undertreated due to several reasons, including denial of the illness as well as cultural and social stigma. With the ubiquity of social media platforms, millions of people are now sharing their online persona by expressing their thoughts, moods, emotions, and even their daily struggles with mental health on social media. Unlike traditional observational cohort studies conducted through questionnaires and self-reported surveys, we explore the reliable detection of depressive …

Analyzing And Learning The Language For Different Types Of Harassment, Mohammadreza Rezvan, Saeedeh Shekarpour, Faisal Alshargi, Krishnaprasad Thirunarayan, Valerie L. Shalin, Amit P. Sheth

Publications

THIS ARTICLE USES WORDS OR LANGUAGE THAT IS CONSIDERED PROFANE, VULGAR, OR OFFENSIVE BY SOME READERS. The presence of a significant amount of harassment in user-generated content and its negative impact calls for robust automatic detection approaches. This requires the identification of different types of harassment. Earlier work has classified harassing language in terms of hurtfulness, abusiveness, sentiment, and profanity. However, to identify and understand harassment more accurately, it is essential to determine the contextual type that captures the interrelated conditions in which harassing language occurs. In this paper we introduce the notion of contextual type in harassment by distinguishing …

Knowledge Infused Learning (K-Il): Towards Deep Incorporation Of Knowledge In Deep Learning, Ugur Kursuncu, Manas Gaur, Amit Sheth

Publications

Learning the underlying patterns in data goes beyondinstance-based generalization to external knowledge repre-sented in structured graphs or networks. Deep learning thatprimarily constitutes neural computing stream in AI hasshown significant advances in probabilistically learning la-tent patterns using a multi-layered network of computationalnodes (i.e., neurons/hidden units). Structured knowledge thatunderlies symbolic computing approaches and often supportsreasoning, has also seen significant growth in recent years,in the form of broad-based (e.g., DBPedia, Yago) and do-main, industry or application specific knowledge graphs. Acommon substrate with careful integration of the two willraise opportunities to develop neuro-symbolic learning ap-proaches for AI, where conceptual and probabilistic repre-sentations are combined. …

Alone: A Dataset For Toxic Behavior Among Adolescents On Twitter, Thilini Wijesiriwardene, Hale Inan, Ugur Kursuncu, Manas Gaur, Valerie L. Shalin, Krishnaprasad Thirunarayan, Amit P. Sheth, I. Budak Arpinar

Publications

The convenience of social media has also enabled its misuse, potentially resulting in toxic behavior. Nearly 66% of internet users have observed online harassment, and 41% claim personal experience, with 18% facing severe forms of online harassment. This toxic communication has a significant impact on the well-being of young individuals, affecting mental health and, in some cases, resulting in suicide. These communications exhibit complex linguistic and contextual characteristics, making recognition of such narratives challenging. In this paper, we provide a multimodal dataset of toxic social media interactions between confirmed high school students, called ALONE (AdoLescents ON twittEr), along with descriptive …

Explainable Ai Using Knowledge Graphs, Manas Gaur, Ankit Desai, Keyur Faldu, Amit Sheth

Publications

During the last decade, traditional data-driven deep learning (DL) has shown remarkable success in essential natural language processing tasks, such as relation extraction. Yet, challenges remain in developing artificial intelligence (AI) methods in real-world cases that require explainability through human interpretable and traceable outcomes. The scarcity of labeled data for downstream supervised tasks and entangled embeddings produced as an outcome of self-supervised pre-training objectives also hinders interpretability and explainability. Additionally, data labeling in multiple unstructured domains, particularly healthcare and education, is computationally expensive as it requires a pool of human expertise. Consider Education Technology, where AI systems fall along a …

Relational Sequential Decision Making, Kaushik Roy

Publications

Markov Decision Processes(MDPs) are the standard for sequential decision making. Comprehensive theory and methods have been developed to deal with solving MDPs in the propositional setting. Real world domains however are naturally represented using objects and relationships. To this effect, relational adaptations of algorithms to solve MDPs have been proposed in recent years. This paper presents a study of these techniques both in the model based and model free setting.