Computer Engineering Commons^™

Deciphering Speech Through Vision: A Deep Learning Lip Reading System, Srujith Rao Ambati

Master's Projects

Lip-reading, a ubiquitous field between computer vision and speech processing, focuses on identifying what spoken words a person generates depending on their uttering lip movements. This paper presents a streamlined lip-reading solution that employs machine learning and deep learning. First, Our work utilizes the Multi-Task Cascaded Convo- lutional Networks to detect facial “landmarks,” including the face and lips region, and the aligns the face. The aligned faces are segmented to get the lip images. Lip images are preprocessed using the Real-Enhanced Super Resolution Generative Adversarial Network to enhance image resolution to identify subtle lip movement in video images: a critical …

Deception Detection Models From Speech, Tien Nguyen

Master's Projects

Recently, researchers have shown an increased interest in automatically detecting deceptive actions. The attention given to this area can be attributed to the many potential applications of deception detection, especially in the field of criminology. To contribute to the deception detection research, this project investigates textual and audio data extracted from spoken and written words. We evaluated and compared the traditional linguistic models with advanced Large Language Models (LLMs) while using Natural Language Processing (NLP) techniques. Additionally, various feature selection techniques were applied to assess the importance of linguistic features. We conducted extensive experiments to evaluate the effectiveness of both …

Multi-Platform Cyberbullying Detection Using Nlp And Machine Learning, Chinmayi Lokeshwar Hegde

Master's Projects

The issue of cyberbullying is growing due to the online anonymity and due to online platforms having less repercussions. This research proposes for proactive measures to detect and prevent such behavior before it reaches the victim. By using data from various social media platforms and employing machine learning techniques, this research proposes an innovative system aimed at identifying and thwarting cyberbullying incidents preemptively. While existing methods have primarily focused on prediction and detection of cyberbullying incidents, there remains a significant gap in research regarding prevention strategies. This project aims to address this gap by leveraging machine learning, natural language processing …

Detecting Crustose Coralline Algae (Cca) In Marine Photos Using Mask R-Cnn, Vrushali Harshwardhan Deshpande

Master's Projects

Coral reefs, made up of thousands of polyps - tiny sac-like marine invertebrates sea anemones and jellyfish, are important to marine ecosystems and prevent loss of life by acting as a natural barrier against storms, floods, and waves. These reefs support a wide range of species, many of which are underexplored and new species being discovered regularly. Crustose coralline algae (CCA) is one of the vital algal species that provides reef structure. Studying the abundance of CCA is important in helping marine biologists analyze coral reef health while understanding the impact of climate change on the marine lifeforms. This study …

Exploring Fluctuations In Working Memory Load Through Pupillometry Using Gabor Image Deletion Tasks, Neenu Antony

Master's Projects

Van der Wel & Van Steenbergen mention that there has been a surge in pupillometry research in the past two decades, particularly in the area of task-evoked pupil dilation in the context of cognitive control tasks. The goal of most of these studies has been focused on finding a link between pupil dilation and effort exerted by an individual [10]. The review by authors Van der Wel & Van Steenbergen, aimed to assess the potential of pupil dilation as an indicator of effort rather than task complexity. Their analysis revealed that heightened task demands in domains such as updating, switching, …

Noteblocklib: A Library For Physics- And Animation-Driven Virtual Midi Instruments For Use In Video Games, Kevin Rotunni

Master's Projects

To better capture the relationship between the performer of a piece of music and the music itself in a video game context, I have designed NoteBlockLib, a system by which MIDI instructions are generated and processed in real time based on the motion and collision data of in-game objects. Ultimately, the movement of instruments made using this system would be driven by the animations of a character in the game. This system would thus allow the player character to interact with the performer in the game without sacrificing the relationship between the performer’s actions and the resulting music or to …

Instagram Data Analysis Using Machine Learning, Lakshmi Prasanna Gorrepati

Master's Projects

With enormous amount of social media content, we can draw valuable insights. In this paper, we apply different Machine Learning and Deep Learning techniques on Instagram data to determine the techniques that work well to discover the engagement class of a social media post. Out of all the social media platforms, Instagram is growing rapidly not just in the number of users but also in terms of Advertisement and marketing surpassing YouTube’s advertisement revenue. The end goal of this paper is to propose a technique to predict the engagement class. We applied Random Forest (RF), Stacking Classifier, Extreme Gradient Boost …

Considering A Unified Model Of Artificial Intelligence Enhanced Social Work: A Systematic Review, Michael Garkish, Lauri Goldkind

Social Service Faculty Publications

Social work, as a human rights–based profession, is globally recognized as a profession committed to enhancing human well-being and helping meet the basic needs of all people, with a particular focus on those who are marginalized vulner- able, oppressed, or living in poverty. Artificial intelligence (AI), a sub-discipline of computer science, focuses on develop- ing computers with decision-making capacity. The impacts of these two disciplines on each other and the ecosystems that social work is most concerned with have considerable unrealized potential. This systematic review aims to map the research landscape of social work AI scholarship. The authors analyzed the …

Few-Shot Learning For Ner Using Maml, Nourchene Bargaoui

Theses and Dissertations

This thesis investigates the application of Few-Shot Learning (FSL) using Model-Agnostic Meta-Learning (MAML) to enhance Named Entity Recognition (NER) within the domain of Natural Language Processing (NLP), specifically focusing on chemical datasets. The primary challenge addressed is the impracticality of relying on extensive annotated datasets, especially in specialized fields like chemistry. The research primarily explores the concept of Few-Shot Learning, aiming to train models on minimal data while maintaining performance across diverse tasks. It delves into the N-way K-shot methodology, where "N" represents the number of classes and "K" signifies the number of examples per class. This approach is further …

Integrating Chatgpt With A-Frame For User-Driven 3d Modeling, Ivan Hernandez

Master's Projects

ChatGPT is a large language model that is capable of creating conversational text and functional code that can be integrated into various technologies, including computer graphics software. Currently, 3D modeling applications can be relatively difficult for novices to learn and understand due to the overwhelming amount of graphical user interfaces. However, we can remedy this issue by leveraging ChatGPT’s conversational language capabilities. Our project described in this report integrates ChatGPT with A-Frame, an online framework for developing virtual reality experiences, to create an immersive and user-friendly 3D modeling environment where users can create and modify 3D models through natural language …

A Framework For Scientific Data Indexing, Searching And Sharing, Apoorva Mohite

Master's Projects

Scientific data continues to grow. Wildfire simulation experiments performed by the WIRC team at SJSU have generated over 138 TB of data so far and it is expected to keep growing. It becomes difficult for researchers to search through that data to find the data of their interest. This data is stored on an HPC cluster that external users do not have access to. The WIRC team also conducts experiments and publishes their research, but the size of data makes it difficult to share these datasets. This project introduces a novel solution to indexing scientific data, searching through the data …

Deep-Learning Approaches To Predict Remaining Useful Life Of Hard Disks, Rohan Mohapatra

Master's Projects

On a daily basis, data centers process huge volumes of data using inexpensive hard disks. Data stored in these disks serve a range of critical functional needs from financial, and healthcare to aerospace. As such, premature disk failure and consequent loss of data can be catastrophic. To mitigate the risk of failures, cloud storage providers perform condition-based monitoring and replace hard disks before they fail. By estimating the remaining useful life (RUL) of hard disk drives, one can predict the time-to-failure of a particular device and replace it at the right time, ensuring maximum utilization whilst reducing operational costs. We …

Adversarial Attacks And Defense Mechanisms In Multivariate Time-Series Forecasting For Applications In Smart And Connected Infrastructures, Pooja Krishan

Master's Projects

The advent of deep learning models has revolutionized the industry over the past decade, leading to the widespread proliferation of smart devices and infrastructures. They play a crucial role in safety-critical applications like self-driving cars and medical image analysis, sustainable technologies like power consumption prediction, and in health monitoring tools to replace industrial equipment like hard disk drives, semiconductor chips, and lithium-ion batteries. But these indispensable deep learning models can be easily fooled to give incorrect predictions with utmost conviction, leading to catastrophic failures in applications where safety is of utmost importance, and resulting in the wastage of resources in …

A Two-Stage Machine Learning Approach For Fake News Detection And News Article Categorization, Snegdha Adusumilli

Master's Projects

This project employs machine learning techniques to develop a sequential model for detecting and categorizing fake news, aiming to mitigate its proliferation in today's digital landscape. The model operates in two phases: in the first phase, the classification algorithms like Naïve Bayes, XGBoost and Random Forest are used to distinguish between true and false news stories and in the second phase the capabilities of Naïve Bayes, XGBoost, Random Forest, and the Transformer-based BERT (Bidirectional Encoder Representations from Transformers) model are leveraged to further categorize the news into specific topics.

The methodology encompasses several key steps: data acquisition, preprocessing, feature extraction, …

Mitigating The Risk Of Reentrancy Attack In Smart Contract Development, Eric Ngo

Master's Projects

Smart contracts, while revolutionizing the blockchain with their immutable nature, are prone to attacks such the reentrancy attack. This attack allows malicious adversaries to repeately enter a contract before previous executions are completed. SpartanScript, a custom dialect of Scheme, is a way for developers to write and develop contracts in an experimental blockchain environment like SpartanGold. Compared to cryptocurrencies that use a virtual machine to run on the blockchain, SpartanScript utilizes a simplified interpreter for rapid prototyping. However, SpartanScript does not have a way to detect and warn developers of reentrancy vulnerabilities. Hence, there is a need to implement reliable …

Secured Data Storage Management With Deduplication In Cloud Computing, Ganesh Regoti

Master's Projects

In the cloud era, cloud storage has become a major service and the security of data and user privacy algorithms are becoming of great importance. This way, we make sure that the encrypted data is kept in the cloud storage. But, the challenges follow: First, storing encrypted data may result in ineffective utilization of cloud resources as in the provision of encrypted data, redundancy cannot be provided. Access control to the encrypted data is difficult as the underlying data is hidden and there is no metric with which the decision to share among users can be easily taken. Deduplication is …

Enhancing Environmental Health And Safety: Fine-Tuning Large Language Models For Domain-Specific Applications, Mohammad Adil Ansari

Master's Projects

This study aims to simplify Environmental Health and Safety (EHS) by leveraging the power of Large Language Models (LLMs). In this research, we focus on fine-tuning three LLMs — LLaMA, Mistral, and Falcon — using PEFT techniques such as QLoRA and SFT, to address domain-specific needs such as safety compliance, incident reporting, and knowledge dissemination. Our research methodology involves fine-tuning each LLM model on a custom dataset compiled from various regulatory agencies, supplemented by targeted web scraping and manual collection of questionnaires to capture and enrich the models with the latest regulations and guidelines. This study aims to compare the …

Fine-Tuning Large Language Models For Folder Structure Generation, Likhith Nemani

Master's Projects

Starting a new project is a significant challenge in the software development world. Building a new project skeleton and configurations will require vast amounts of time and effort. This project aims to overcome the difficulty presented by this challenge using advanced large language models, specifically fine-tuning LLMs. Our initial focus with the implementation is to use the powerful capabilities of advanced modern models to simplify and accelerate the complicated process of getting new projects started. The solution process begins with a user posting a README file to a predetermined repository. This README file then is used as a source for …

Resume Content Generation Using Llama 2 With Adapters, Navaneeth Sai Nidadavolu

Master's Projects

The primary objective of this project is to optimize the Llama language model to generate customized resumes containing domain-specific job descriptions and maintain the linguistic capabilities of the large language model. Building upon the prior research by Sumed Kale on Job Tailored Resume content generation using GPT-2, where he employed full fine-tuning of the model and demonstrated the capability of LLMs to generate resume content, it is evident that while effective, full fine-tuning has its limitations. Primarily, it is computationally expensive, which can pose constraints, especially for large models. Additionally, during the fine-tuning process, there is a risk of losing …

Enhancing Restaurant Sales Prediction: The Dynamic Forecasting Engine, Rahul Sanjay Morishetti

Master's Projects

This project introduces a "dynamic forecasting engine," designed to transform the way restaurants predict sales. The engine dynamically handles seasonal ARIMA_HoltWinter hybrid model, XGBoost, and LSTM algorithms to dynamically select the best forecasting method based on data volume, variety, and customer taste preferences delving upon the spice level categorical sales. This guide differs from traditional crystal ball approaches because it has the ability to improve over time as new data comes in terms of spice levels. It emphasizes the importance of dataset size in the selection of machine learning algorithms through complexity for large datasets and simplicity for smaller ones …

Domain Expert Bot, Amrutha Dondemadahalli Ramegowda

Master's Projects

The fast growth of artificial intelligence in human-computer interaction has been aided significantly by the introduction of conversational AI systems. This project presents a Domain Expert Bot, a multi-domain conversational bot built with advanced NLP techniques incorporated through Sentence-BERT and MapReduce to allow the bot to analyze and comprehend challenging user queries on various topics. The bot can converse on different subjects ranging from technology topics to healthcare, environment, politics, and casual discussions. It excels in understanding deep language contexts and efficiently processes large datasets, ensuring prompt and accurate responses. Furthermore, it uses advanced ranking algorithms to perform real- time …

Enhancing Medical Chatbots With Image Diagnosis, Swatisri Chavali

Master's Projects

Medical chatbots, at the conjunction of artificial intelligence and healthcare, are the very cornerstone of a transformative force in diagnostic capabilities and communication channels for healthcare professionals. The history of this journey, from early chatbot models to sophisticated systems, is born out of a relentless pursuit of accuracy and contextual understanding. This proposal acknowledges the critical role played by NLTK in raising the interpretability and communicative capabilities of intelligent systems, meeting challenges that arise from varying writing styles and accommodating the standards of the medical field. The integration of NLTK is a linchpin, bridging the gap between sophisticated technological architectures …

Parallel Powerplay: Optimizing Performance With Mapreduce And Kubernetes Fusion, Shradha Chaturvedi

Master's Projects

The combination of MapReduce (MR) & Kubernetes (K8s) strengths is not explored, and this study leverages the synergy between the two frameworks to meet the growing demands of data-intensive applications. First, this report elaborates on the existing literature work to understand the pros and cons of using MR and K8s, in what use cases these frameworks come to use, and investigates the effectiveness of research studies that explore the combination. This study aims to research the efficacy of the fusion of MR and K8s, considering these factors - application use case, infrastructure design, resource allocation, load balancing, and hypertuning parameters …

Satellite Handover Optimization Using Predicted Satellite-To-Base-Station Proximity, Pranathi Kunadi

Master's Projects

Modern telecommunications heavily rely on Satellite communication networks to provide global coverage, especially in remote areas which link the whole world in a loop. Conventional handover algorithms methods rely on fixed and predefined rules and thresholds predefined statically to make a handover decision. However, these static handover algorithms may become inefficient under the changing conditions of the network. Therefore, it would be useful to measure the proximity order of satellites to the specific base station. Consequently, the assessed relative proximity helps in optimizing the handovers proactively inside related coverage areas. This results in the service quality and the delays in …

Personalized Medical Predictions, Bhargavi Chevva

Master's Projects

Over the past few years, personalized medicine has gained traction due to its ability to solve medical issues efficiently for a person based on their personal characteristics. This project aims to design a machine-learning model that can generate predictions of lab test scores in the future based on past medical history. The model is trained using the MIMIC-4 (Medical Information Mart for Intensive Care) dataset that consists of medical records of over 40,000 patients. The proposed model, MOE-BEHRT, consists of Bidirectional Encoder Representations from Transformers on Electronic Health Records (BEHRT) and Mixture of Experts (MOE). The BEHRT model was originally …

Credit Score-Based Lending System On The Ethereum Platform, Mayuri Shimpi

Master's Projects

Traditional banking systems act as intermediaries, assessing risks and profiting from interest rate differentials. Credit scores, provided by trusted bureaus, are commonly used to evaluate the creditworthiness of borrowers. Cryptocurrencies have emerged as a significant and innovative medium due to their decentralized nature, operating without reliance on a central authority, such as a government.

This report describes a project to implement the Autonomous Lending system on the Ethereum Platform (ALOE), as proposed in [1], aiming to seamlessly integrate traditional credit scoring methodologies for evaluating a borrower's risk of default. The objective of this project report is to establish a robust …

Advancing Phishing Protection: Employing Sophisticated Methods For Precise Url Evaluation, Abbhinav Reddie Nomuiia

Master's Projects

In short, the incidence of phishing - the illegal act of people pretending to be well-known companies to secure personal information - has skyrocketed in the past few years. In 2022 alone, 300,000 consumers in the United States were captured by scammers using phishing techniques, losing in all over $50 million. In the span of two weeks, over 510 million attempts occurred in a variety of sectors, particularly instant messaging platforms, package delivery businesses, and digital currency trading. Since most businesses have recognized that they are prone to these exposure cases, there has been a sixty percent increase in businesses …

Novel Approach To Music Analysis Using Apache Spark, Nidhi Zare

Master's Projects

Music is one of the most common source of entertainment. Every user has their own taste of music and prefer to listen music that adheres to their taste and mood. There are various categories, called as music genres in which music can be classified. This research project addresses the challenge in music genre classification by using various deep learning models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Very Deep Convolutional Networks (VGGNet), ResNet and others. The primary objective of this research is to enhance the accuracy of music genre classification using a distributed computing framework Apache Spark. …

Birdsong Classification Using Deep Learning And Mixit, Sasanka Kosuru

Master's Projects

The identification of bird species using deep learning techniques presents a novel approach in bioacoustics, by significantly advancing our understanding and enhancing our capabilities in bird species recognition from audio recordings. The value of audio over visual data for monitoring ecological patterns in birds can be highlighted with the deployment of automated recording devices in remote wildlife sensing, offering a more cost-effective, non-invasive, and practical solution. However, the methods of processing and classifying the audio remain challenging due to the complexity of bird audio, characterized by diverse vocalizations and imminent environmental noise, which poses difficult challenges to perform effective classification. …

Optimization Of Inter-Satellite Routing Using Lstm-Based Path Prediction Model, Yash Bhamare

Master's Projects

Satellite networks are one of the most important components that fulfill the world’s need for connectivity. To ensure that communication is efficient and reliable, robust routing algorithms are a must. Because, although it is true that certain routing characteristics may not be permanently and continuously flawless, a routing technique must effectively adapt to modifications in such network characteristics. The new routing method uses a Long Short-Term Memory (LSTM) model to manage dynamic metrics for Low Earth Orbit satellite networks. This LSTM model is aimed at predicting the optimal routing direction on the premise that a satellite is soon to be, …