Data Science Commons^™

A Comparison Of Machine Learning Techniques For Validating Students’ Proficiency In Mathematics, Alexander Avdeev

Dissertations, Theses, and Capstone Projects

A principal goal of this project was to compare several machine learning (ML) algorithms to explore and validate math proficiency classifications based on standardized test scores. The data used in these analyses came from the 6th-grade students’ mathematics assessment records of the New York State Education Department’s Testing Program (NYSTP). Our approach was to test a number of competing machine learning (ML) algorithms for classifying students’ as proficient based on their test scores and other demographic information. Our samples were drawn from the 2016 test-taking cohort of 6th-grade students (N=156,800). Five classifiers including multinominal logistic regression (MLR), XGBoost, Tree-As, Lagrangian …

Symmetry-Inspired Analysis Of Biological Networks, Ian Leifer

Dissertations, Theses, and Capstone Projects

The description of a complex system like gene regulation of a cell or a brain of an animal in terms of the dynamics of each individual element is an insurmountable task due to the complexity of interactions and the scores of associated parameters. Recent decades brought about the description of these systems that employs network models. In such models the entire system is represented by a graph encapsulating a set of independently functioning objects and their interactions. This creates a level of abstraction that makes the analysis of such large scale system possible. Common practice is to draw conclusions about …

Heating Fire Incidents In New York City, Merissa K. Lissade

Dissertations, Theses, and Capstone Projects

If you have ever had the Citizen app downloaded on your smart phone, then you know how many alerts you receive in a day living in New York City (NYC). Citizen is a mobile app that sends real-time safety alerts based on the location of its user. In my experience having the app, I have seen many notifications of fires caused by heaters during the winter. On the morning of January 9^th, 2022, I received a notification of an accidental blaze that took the lives of 17 people from the choking smoke of a 19-story residential building in …

Why, New York City? Gauging The Quality Of Life Through The Thoughts Of Tweeters, Sheryl Williams

Dissertations, Theses, and Capstone Projects

As a resource for social data, Twitter’s platform has been used to measure the quality of life through sentiment analysis. This capstone project explores another methodological technique—querying Twitter data around specific keyword terms to determine dominant topics, word patterns, and sentiment leanings in a geographical area. Focusing on New York City and Los Angeles for comparative analysis, the keyword term “why” will be used to build a Python analysis around topic modeling and sentiment analysis. Using this approach, the analysis reveals social and cultural differences, the overall sentiment of tweets, and subjects of interest to tweeters.

GitHub Repository for all …

Exploring The Effectiveness Of Multiple-Exemplar Training For Visual Analysis Of Ab-Design Graphs, Verena S. Bethke

Dissertations, Theses, and Capstone Projects

In behavior analysis, data are usually analyzed using visual analysis of the graphed data. There are a wide range of methods used to visually analyze data, from a basic ‘textbook’ style approach to the use of visual aids, decision-rubrics, and computer-based approaches. In the literature, there have been some comparisons of the efficacy of different approaches. Visual analysis as a behavior can be taught using a variety of methods, independent of how the skill itself is to be performed. Teaching methods include lecture, online instruction, and equivalence-based instruction. There is not much research on the teaching of visual analysis specifically, …

Representation Learning For Chemical Activity Predictions, Mohamed S. Ayed

Dissertations, Theses, and Capstone Projects

Computational prediction of a phenotypic response upon the chemical perturbation on a biological system plays an important role in drug discovery and many other applications. Chemical fingerprints derived from chemical structures are a widely used feature to build machine learning models. However, the fingerprints ignore the biological context, thus, they suffer from several problems such as the activity cliff and curse of dimensionality. Fundamentally, the chemical modulation of biological activities is a multi-scale process. It is the genome-wide chemical-target interactions that modulate chemical phenotypic responses. Thus, the genome-scale chemical-target interaction profile will more directly correlate with in vitro and in …

Blockchain: Key Principles, Nadezda Chikurova

Dissertations, Theses, and Capstone Projects

“Blockchain: Key Principles” is an interactive visual project that explains the importance of data privacy and security, decentralized computing, and open-source software in the modern digital world through the history of the underlying principles of blockchain technology. Some of these key concepts have their roots in the time before the Information Age. By explaining the history of these principles, I want to present the fact that over the past centuries, humanity has been fighting for their privacy, security, and the ability to efficiently express themselves one way or another. Blockchain technology, which was introduced to the public in 2008 through …

Air Pollution, Climate Change, And Our Health, Kathia Vargas Feliz

Dissertations, Theses, and Capstone Projects

Climate change is a subject that is creating a lot of controversies nowadays. From newspapers to researchers, there are big efforts going on trying to bring awareness about the effects of air pollution and climate change over time. It is recommended that governments all over the world, and people from all communities act by taking care of the environment because the situation might turn out to be irremediable. There is a quote by Leonardo Dicaprio stating, “Climate change is real. It is happening right now; it is the most urgent threat facing our entire species and we need to …

Liquidity Commonality With Factor Models, Ernesto Garcia Iii

Dissertations, Theses, and Capstone Projects

Market microstructure research has recently devoted attention to a phenomenon called commonality in liquidity. In this dissertation, I will analyze commonality in liquidity using a novel factor model approach and a generalized definition of commonality in liquidity. This analysis will show that commonality in liquidity is rarely a marketwide phenomenon and is mostly restricted to stocks with a large market capitalization. Additionally, commonality in liquidity is a very recent phenomenon whose appearance coincides with a rise in passive investing after the Dotcom Bubble burst and, more so, after the 2008 Financial Crisis. I will present evidence that suggests commonality in …

A Citizen-Science Approach For Urban Flood Risk Analysis Using Data Science And Machine Learning, Candace Agonafir

Dissertations and Theses

Street flooding is problematic in urban areas, where impervious surfaces, such as concrete, brick, and asphalt prevail, impeding the infiltration of water into the ground. During rain events, water ponds and rise to levels that cause considerable economic damage and physical harm. The main goal of this dissertation is to develop novel approaches toward the comprehension of urban flood risk using data science techniques on crowd-sourced data. This is accomplished by developing a series of data-driven models to identify flood factors of significance and localized areas of flood vulnerability in New York City (NYC). First, the infrastructural (catch basin clogs, …

Messiness: Automating Iot Data Streaming Spatial Analysis, Christopher White, Atilio Barreda Ii

Publications and Research

The spaces we live in go through many transformations over the course of a year, a month, or a day; My room has seen tremendous clutter and pristine order within the span of a few hours. My goal is to discover patterns within my space and formulate an understanding of the changes that occur. This insight will provide actionable direction for maintaining a cleaner environment, as well as provide some information about the optimal times for productivity and energy preservation.

Using a Raspberry Pi, I will set up automated image capture in a room in my home. These images will …

Analyze And Examine Wildfire Events In California, Aleena Hoodith, Sakim Zaman, Safoan Hossain, Jiehao Huang

Publications and Research

•A wildfire is an unplanned, unwanted, uncontrolled fire in an area of combustible vegetation starting in rural areas and urban areas. •Recent studies have shown that the effect of anthropogenic climate change has fueled the wildfire events, leading to an increase in the annual burned areas and number of events. •California is one of the places having the most deadliest and destructive wildfire seasons. With the global warming effect of 1°C since 1850, the 20 largest wildfires events that have occurred in California, 8 of them were in 2017. (Center For Climate And Energy Solutions) •Climate change is primarily caused …

Leveraging The Popularity Of Virtual Conferencing Due To The Covid-19 Pandemic To Create New Opportunities For Stem Education, Andrew Singh, Nazrul I. Khandaker, Violeta Escandon Correa, Omadevi Singh, Ariel Skobelsky, Farhan Tanvir, Brian Sukhnandan, Matthew Khargie, Elton Selby, Masud Ahmed

Publications and Research

Due to the COVID-19 pandemic, virtual learning has become a necessity for K9-16 education. Virtual classwork has been administered through platforms such as Google Classroom, Clever, and iReady. During the summer of 2021, the City University of New York (C.U.N.Y) York College campus hosted its NASA MAA MUREP (Minority University Research and Education Project Aerospace Academy) program virtually using a combination of Zoom, Google Docs, and even Canva, which some students requested as a more intuitive alternative to Microsoft PowerPoint. Students were mentored to use the scientific method to explore their interests in the STEM field, with a geoscience or …

Addressing The Learning Loss During The Covid-19 Pandemic Through The Adaptation Of Virtual Platforms, Nazrul I. Khandaker, Anika Nawar Mayeesha, Violeta Escandon Correa, Toralv Munro, Andrew Singh, Matthew Khargie, Ality Aghedo, Jasmin Budhan, Krishna Mahabir, Belal A. Sayeed

Publications and Research

The York College-hosted NASA MAA (MUREP AEROSPACE ACADEMY) has always played a pivotal role in minimizing the learning loss during the summer months, which was heightened during the pandemic. Support from AT&T, Con Edison and NASA enabled the MAA program at York College to offer a virtual STEM education with an earth science concentration to 1000 plus underserved K1-12 students from the community last summer, including 160 high school students. Two factors made this endeavor fruitful: allowing additional time to engage in STEM lessons and increasing self-motivation to successfully accomplish assigned tasks. Students built partnerships and resolved technical issues with …

Piecewise Linear Manifold Clustering, Artyom Diky

Dissertations, Theses, and Capstone Projects

This work studies the application of topological analysis to non-linear manifold clustering. A novel method, that exploits the data clustering structure, allows to generate a topological representation of the point dataset. An analysis of topological construction under different simulated conditions is performed to explore the capabilities and limitations of the method, and demonstrated statistically significant improvements in performance. Furthermore, we introduce a new information-theoretical validation measure for clustering, that exploits geometrical properties of clusters to estimate clustering compressibility, for evaluation of the clustering goodness-of-fit without any prior information about true class assignments. We show how the new validation measure, when …

Detecting Stance On Covid-19 Vaccine In A Polarized Media, Rodica Ceslov

Dissertations, Theses, and Capstone Projects

The growing polarization in the United States has been widely reported. Media coverage plays an important role in shaping public opinion and influences public debates on complex and unfamiliar topics. There are some benefits to individuals and society from political polarization and conflict between opposing viewpoints. However, recent research has primarily highlighted the negative consequences of polarization which reached an all-time high. One such topic is the Covid-19 vaccine which was developed in record time, and the public learned about its safety and possible risks through the media coverage.

In this capstone, we examine U.S. news media coverage on the …

Making Space For Unquantifiable Data: Hand-Drawn Data Visualization, Eva Sibinga

Dissertations, Theses, and Capstone Projects

This project makes space for personal “data” around labor and care, prompting users to consider the concrete and abstract (quantifiable and unquantifiable) forms labor and care take in their lives. The interactive, subjective data visualization uses hand-drawn visual elements to foreground that data about care and human interaction will always be ambiguous and complex, that they may never be satisfactorily or universally quantified, and that they will always be out of reach of perfect categorization.

The project provides an alternative to prescriptive truth-telling with data. Instead of using a dataset to provide data-driven answers and insights to users, the interactive …

Teaching Machine Learning For The Physical Sciences: A Summary Of Lessons Learned And Challenges, Viviana Acquaviva

Publications and Research

This paper summarizes some challenges encountered and best practices established in several years of teaching Machine Learning for the Physical Sciences at the undergraduate and graduate level. I discuss motivations for teaching ML to physicists, desirable properties of pedagogical materials, such as accessibility, relevance, and likeness to real-world research problems, and give examples of components of teaching units.

Content Analysis Of Two-Year And Four-Year Data Science Programs In The United States, Elizabeth Milonas, Duo Li, Qiping Zhang

Publications and Research

Data has grown exponentially in the last decade, and this growth has resulted in vast challenges for both business and IT domains (Hassan & Liu, 2019). This growth has given rise to the Data Science field, which has also grown exponentially in the last few years (Hassan & Liu, 2019; Song & Zhu, 2016). The Data Science field has its origins in the statistics and mathematics domain (Cao, 2017b), but is now considered a multidisciplinary field (Aasheim et al., 2015). Data Science warrants knowledge of data analytics, programming, systems, applications, informatics, computing, communication, management, and sociology (Aasheim et al., 2015; …

Using Data Science To Create An Impact On A City Life And To Encourage Students From Underserved Communities To Get Into Stem, Elena Filatova, Deborah Hecht

Publications and Research

In this paper, we introduce a novel methodology for teaching Data Science. Our methodology relies on the outlook of the student body in our college. Our college is an urban, commuter, HSI (Hispanic Serving Institution) school with 34% Hispanic and 29% Black students. 61% of our students come from households with an income of less than $30,000+. Thus, many students in our college come from the communities that are underrepresented in the STEM fields and in the decision-making positions in the government (on the city level, state level, country level). However, in our methodology, we want to flip the situation …

---

Go to article

Learn Biologically Meaningful Representation With Transfer Learning, Di He

Dissertations, Theses, and Capstone Projects

Machine learning has made significant contributions to bioinformatics and computational biol­ogy. In particular, supervised learning approaches have been widely used in solving problems such as bio­marker identification, drug response prediction, and so on. However, because of the limited availability of comprehensively labeled and clean data, constructing predictive models in super­ vised settings is not always desirable or possible, especially when using data­hunger, red­hot learning paradigms such as deep learning methods. Hence, there are urgent needs to develop new approaches that could leverage more readily available unlabeled data in driving successful machine learning ap­ plications in this area.

In my dissertation, …

An Empirical Study Of Refactorings And Technical Debt In Machine Learning Systems, Yiming Tang, Raffi Khatchadourian, Mehdi Bagherzadeh, Rhia Singh, Ajani Stewart, Anita Raja

Publications and Research

Machine Learning (ML), including Deep Learning (DL), systems, i.e., those with ML capabilities, are pervasive in today’s data-driven society. Such systems are complex; they are comprised of ML models and many subsystems that support learning processes. As with other complex systems, ML systems are prone to classic technical debt issues, especially when such systems are long-lived, but they also exhibit debt specific to these systems. Unfortunately, there is a gap of knowledge in how ML systems actually evolve and are maintained. In this paper, we fill this gap by studying refactorings, i.e., source-to-source semantics-preserving program transformations, performed in real-world, open-source …

Estimation Of The Planetary Boundary Layer Height: Part 1: Global Radar Wind Profiler Network Data; Part 2: A Comparison To Ceilometer Data, Holly Josephs

Theses and Dissertations

Two methods for estimating the planetary boundary layer, an algorithm to identify a maximum in the backscatter and a covariance wavelet transform method, are explored and applied to global radar wind profiler network data and ceilometer data respectively. The objective of the study is to establish that the data sources and algorithms can be used to estimate planetary boundary layer heights so that global studies can make use of these estimates. Data from the global network of wind profilers required significant restructuring and quality control in order to be used for the present study. The maximum backscatter identification algorithm was …

Application Of Randomness In Finance, Jose Sanchez, Daanial Ahmad, Satyanand Singh

Publications and Research

Brownian Motion which is also considered to be a Wiener process and can be thought of as a random walk. In our project we had briefly discussed the fluctuations of financial indices and related it to Brownian Motion and the modeling of Stock prices.

Discovering Kepler’S Third Law From Planetary Data, Boyan Kostadinov, Satyanand Singh

Publications and Research

In this data-inspired project, we illustrate how Kepler’s Third Law of Planetary Motion can be discovered from fitting a power model to real planetary data obtained from NASA, using regression modeling. The power model can be linearized, thus we can use linear regression to fit the model parameters to the data, but we also show how a non-linear regression can be implemented, using the R programming language. Our work also illustrates how the linear least squares used for fitting the power model can be implemented in Desmos, which could serve as the computational foundation for this project at a lower …

A New Feature Selection Method Based On Class Association Rule, Sami A. Al-Dhaheri

Dissertations, Theses, and Capstone Projects

Feature selection is a key process for supervised learning algorithms. It involves discarding irrelevant attributes from the training dataset from which the models are derived. One of the vital feature selection approaches is Filtering, which often uses mathematical models to compute the relevance for each feature in the training dataset and then sorts the features into descending order based on their computed scores. However, most Filtering methods face several challenges including, but not limited to, merely considering feature-class correlation when defining a feature’s relevance; additionally, not recommending which subset of features to retain. Leaving this decision to the end-user may …

Public Interest Technology – Exploring Covid-19 Health Data, Sarah Zelikovitz

Open Educational Resources

This module is part of a Introduction to Data Science course that covers the different parts of the data science process: data acquisition, cleaning, exploratory data analysis, and modeling. The COVID-19 pandemic has created much interest in public health data, as well as interest in visualization of all types of data. Public health data has a set of challenges that is unique to health data, with HIPAA laws, and real time collection of data. With COVID-19, the challenges are particularly amplified, as data collection and statistics collected are constantly changing in response to feedback from labs, hospitals, drug companies, and …

Goes-R Supervised Machine Learning, Ronald Adomako

Dissertations and Theses

The GOES-R series is a product line of four satellite, with two currently on-orbit (GOES-16 “East” and GOES-17 “West”). GOES-17 is susceptible to a Loop-Heat-Pipe (LHP) phenomenon where during Fall and Spring seasons, there are times of day where some of the infrared bands records inaccurate readings from the Advanced Baseline Imager (ABI). This occurs from joint astronomical behavior and position of the GOES-17. This calibration issue occurs when the LHP instrument fails to radiate the heat of the sun out of ABI. Predictive Calibration (pCal) is an algorithm developed by instrument vendors for the National Oceanic Atmospheric Agency (NOAA) …

Principal Component Analysis For Predicting The Party Of The Legislators, Afsana Mimi

Publications and Research

In Spring 2020, I did a project, "Decision Tree Predicting the Party of Legislators," and construct a decision tree model to predict legislators' parties' based on their votes. We also use this model to identify legislators who frequently voted against their parties. We used the legislators' roll call votes, Office of Clerk U.S. House of Representatives Data Sets (Categorical values) collected in 2018 and 2019. In this new project, We study the 2018 and 2019 vote data using Principal Component Analysis (PCA). The goal is to find a (compressed) model using unsupervised learning to distinguish the legislators' parties, and PCA …

Open Data, Collaborative Working Platforms, And Interdisciplinary Collaboration: Building An Early Career Scientist Community Of Practice To Leverage Ocean Observatories Initiative Data To Address Critical Questions In Marine Science, Robert M. Levine, Kristen E. Fogaren, Johna E. Rudzin, Christopher J. Russoniello, Dax C. Soule, Justine M. Whitaker

Publications and Research

Ocean observing systems are well-recognized as platforms for long-term monitoring of near-shore and remote locations in the global ocean. High-quality observatory data is freely available and accessible to all members of the global oceanographic community—a democratization of data that is particularly useful for early career scientists (ECS), enabling ECS to conduct research independent of traditional funding models or access to laboratory and field equipment. The concurrent collection of distinct data types with relevance for oceanographic disciplines including physics, chemistry, biology, and geology yields a unique incubator for cutting-edge, timely, interdisciplinary research. These data are both an opportunity and an incentive …