Data Storage Systems Commons^™

Performance Of Inline Compression With Software Caching For Reducing The Memory Footprint In Pysdc, Emily Lattanzio

All Theses

The volume of data required for High Performance Computing (HPC) applications is growing faster than the memory storage available to store the required data, leading to performance bottlenecks in transferring data. Whether sending data from main memory to computation nodes or between parallel processes during runtime, the more data there is to send, the longer it will take to for that data to be sent from one location to the next. Hence the need for inline data compression, which reduces the amount of allocated memory needed by storing the largest data structures in a compressed format and decompressing/recompressing single variables …

Blockchain-Integrated Version Control For Secure And Transparent Software Supply Chains, Iwinosa W. Aideyan

All Theses

The software supply chain encompasses all stages of software development and delivery from initial coding and version control to integration and deployment. As development environments become increasingly distributed and reliant on external dependencies, ensuring the integrity, auditability, and consistency of code changes has become a pressing challenge. Traditional version control systems like Git, while effective for collaboration and tracking revisions, do not inherently provide tamper-evident commit histories. Features such as history rewriting (e.g., git rebase, git push --force) can be exploited to manipulate commit logs without detection, posing risks in security-sensitive domains. This thesis proposes a blockchain-integrated version control framework …

Optimizing Compression Efficiency With Adaptive Quantization Bit Depths, Carson Sisk

All Theses

Large-scale scientific instruments and applications generate massive amounts of data, lead- ing to significant challenges in data transfer and storage for analysis. This constitutes a major

bottleneck to workflow efficiency and scientific throughput. Lossy compression offers a solution to

these storage challenges in increasingly complex systems and services. Error-bounded lossy compression allows users to limit the error introduced during the compression process according to a user-defined metric and achieves significantly higher compression ratios than lossless compression for floating-point data. However, certain data types and compression configurations hinder the attainment of large compression ratios. To address the need for improved compression …

Application Of Lossy And Lossless Compression To Dicom Files, Yizhe Yang

All Theses

The Digital Imaging and Communications in Medicine (DICOM) standard is widely utilized for the management, storage, and transfer of medical images. However, the substantial file sizes associated with DICOM data present challenges in terms of storage and data transmission. Data reduction techniques help address these challenges by minimizing the size of the data while preserving its integrity. This thesis examines various compression methods aimed at reducing the size of DICOM files. We evaluate five lossless compressors and four lossy compressors on DICOM data to compare and assess their performance. Through an analysis of each compressor’s compression efficiency and resulting image …

Analysis Of Cnn Performance Utilizing Jpeg Compressed Images Created On An Fpga, Timothy Shaughnessy

All Theses

JPEG (Joint Photographic Experts Group) was formed in 1986 to create a method to reduce image size primarily for ease of transfer on the Internet. Released to the public in 1992, JPEG compression is a form of lossless compression that has been a staple for compressing images. JPEG is the go-to image compressor because it provides high compression ratios while maintaining visual integrity for the human eye. Growing image sizes have made JPEG compression increasingly relevant. It is vital to keep up with growing data sizes for improved image handling performance on an edge device like a Field-Programmable Gate Array …

Analyzing An In-Line Compression Management System For Improved Performance In A High-Performance Computing Environment, Steven Platt

All Theses

High-performance computing (HPC) has enabled advancements in computation speed and resource cost by utilizing all available server resources and using parallelization for speedup. This computation scheme encourages simulation model development, massive data collection, and AI computation models, all of which store and compute on massive amounts of data. Data compression has enhanced the performance of storing and transferring this HPC application data to enable acceleration, but the benefits of data compression can also be transferred to the active allocated memory used by the application. In-line compression is a compression method that keeps the application memory compressed in allocated memory, decompressing …

Performance Modeling Of Inline Compression With Software Caching For Reducing The Memory Footprint In Pysdc, Sansriti Ranjan

All Theses

Modern HPC applications compute and analyze massive amounts of data. The data volume is growing faster than memory capabilities and storage improvements leading to performance bottlenecks. An example of this is pySDC, a framework for solving collocation problems iteratively using parallel-in-time methods. These methods require storing and exchanging 3D volume data for each parallel point in time. If a simulation consists of M parallel-in-time stages, where the full spatial problem has to be stored for the next iteration, the memory demand for a single state variable is M ×Nx ×Ny ×Nz per time-step. For an application simulation with many state …

Scalable Data-Driven Predictive Modeling And Analytics For Cho Process Development Optimization, Sarah Mbiki

All Dissertations

In 1982, the FDA approved the first recombinant therapeutic protein, and since then, the biopharmaceutical industry has continued to develop innovative and highly effective biological drugs for various illnesses¹. These drugs are produced using host organisms that are modified to hold the genetic encoding of the targeted protein¹. Of the many host organisms, Chinese hamster ovary (CHO) cells are often used due to capability to perform posttranslational modification (PTM): which allows human-like synthesis of proteins unlikely to invoke immunogenicity in humans ^1,2.

Despite all the positive attributes, many challenges are associated with CHO cell cultures, …

Application Of Image Processing And Convolutional Neural Networks For Flood Image Classification And Semantic Segmentation, Jaku Rabinder Rakshit Pally

All Theses

Floods are among the most destructive natural hazards that affect millions of people across the world leading to severe loss of life and damage to property, critical infrastructure, and the environment. Deep learning algorithms are exceptionally valuable tools for collecting and analyzing the catastrophic readiness and countless actionable flood data. Convolutional neural networks (CNNs) are one form of deep learning algorithms widely used in computer vision which can be used to study flood images and assign learnable weights and biases to various objects in the image. Here, we leveraged and discussed how connected vision systems can be used to embed …