Physical Sciences and Mathematics Commons^™

Enhancing Research Productivity: Seamless Integration Of Personal Devices And Hpc Resources With The Cybershuttle Notebook Gateway, Yasith Jayawardana, Dimuthu Wannipurage, Eroma Abeysinghe, Suresh Marru

Computer Science Faculty Publications

Scientists often utilize personal laptops and workstations for initial research stages and turn to high-performance computing (HPC) supercomputers for compute-intensive tasks. However, seamless transitions between these environments are vital for enhancing productivity and accelerating research progress. Our paper presents the Cybershuttle Notebook Gateway, an open-source framework crafted to streamline this transition, optimize resource utilization, and reduce time-to-science for researchers. Leveraging JupyterLab, the framework extends kernel mechanics for seamless provisioning and connection to remote HPC cluster kernels. We delve into its architecture, which separates user authentication, kernel provisioning, and remote file system access. Additionally, we highlight practical capabilities like analyzing network …

Interactive Data Analysis Of Multi-Run Performance Data, Vanessa Lama

Masters Theses

Multi-dimensional performance data analysis presents challenges for programmers, and users. Developers have to choose library and compiler options for each platform, analyze raw performance data, and keep up with new technologies. Users run codes on different platforms, validate results with collaborators, and analyze performance data as applications scale up. Site operators use multiple profiling tools to optimize performance, requiring the analysis of multiple sources and data types. There is currently no comprehensive tool to support the structured analysis of unstructured data, when holistic performance data analysis can offer actionable insights and improve performance. In this work, we present thicket, a …

Evaluation Of Distributed Programming Models And Extensions To Task-Based Runtime Systems, Yu Pei

Doctoral Dissertations

High Performance Computing (HPC) has always been a key foundation for scientific simulation and discovery. And more recently, deep learning models' training have further accelerated the demand of computational power and lower precision arithmetic. In this era following the end of Dennard's Scaling and when Moore's Law seemingly still holds true to a lesser extent, it is not a coincidence that HPC systems are equipped with multi-cores CPUs and a variety of hardware accelerators that are all massively parallel. Coupling this with interconnect networks' speed improvements lagging behind those of computational power increases, the current state of HPC systems is …

Holistic Performance Analysis And Optimization Of Unified Virtual Memory, Tyler Allen

All Dissertations

The programming difficulty of creating GPU-accelerated high performance computing (HPC) codes has been greatly reduced by the advent of Unified Memory technologies that abstract the management of physical memory away from the developer. However, these systems incur substantial overhead that paradoxically grows for codes where these technologies are most useful. While these technologies are increasingly adopted for use in modern HPC frameworks and applications, the performance cost reduces the efficiency of these systems and turns away some developers from adoption entirely. These systems are naturally difficult to optimize due to the large number of interconnected hardware and software components that …

Load Balancing Algorithms For Parallel Spatial Join On Hpc Platforms, Jie Yang

Dissertations (1934 -)

Geospatial datasets are growing in volume, complexity, and heterogeneity. For efficient execution of geospatial computations and analytics on large scale datasets, parallel processing is necessary. To exploit fine-grained parallel processing on large scale compute clusters, partitioning of skewed datasets in a load-balanced way is challenging. The workload in spatial join is data dependent and highly irregular. Moreover, wide variation in the size and density of geometries from one region of the map to another, further exacerbates the load imbalance. This dissertation focuses on spatial join operation used in Geographic Information Systems (GIS) and spatial databases, where the inputs are two …

A Practical Approach To Automated Software Correctness Enhancement, Aleksandr Zakharchenko

Dissertations

To repair an incorrect program does not mean to make it correct; it only means to make it more-correct, in some sense, than it is. In the absence of a concept of relative correctness, i.e. the property of a program to be more-correct than another with respect to a specification, the discipline of program repair has resorted to various approximations of absolute (traditional) correctness, with varying degrees of success. This shortcoming is concealed by the fact that most program repair tools are tested on basic cases, whence making them absolutely correct is not clearly distinguishable from making them relatively more-correct. …

Quantum Simulation Using High-Performance Computing, Collin Beaudoin, Christian Trefftz, Zachary Kurmas

Masters Theses

Hermitian matrix multiplication is one of the most common actions that is performed on quantum matrices, for example, it is used to apply observables onto a given state vector/density matrix.

ρ→Hρ

Our goal is to create an algorithm to perform the matrix multiplication within the constraints of QuEST [1], a high-performance simulator for quantum circuits. QuEST provides a system-independent platform for implementing and simulating quantum algorithms without the need for access to quantum machines. The current implementation of QuEST supports CUDA, MPI, and OpenMP, which allows programs to run on a wide variety of systems.

Statistical Modeling Of Hpc Performance Variability And Communication, Jered B. Dominguez-Trujillo

Computer Science ETDs

Understanding the performance of parallel and distributed programs remains a focal point in determining how compute systems can be optimized to achieve exascale performance. Lightweight, statistical models allow developers to both characterize and predict performance trade-offs, especially as HPC systems become more heterogeneous with many-core CPUs and GPUs. This thesis presents a lightweight, statistical modeling approach of performance variation which leverages extreme value theory by focusing on the maximum length of distributed workload intervals. This approach was implemented in MPI and evaluated on several HPC systems and workloads. I then present a performance model of partitioned communication which also uses …

Efficient Filters For Geometric Intersection Computations Using Gpu, Yiming Liu, Satish Puri

Computer Science Faculty Research and Publications

Geometric intersection algorithms are fundamental in spatial analysis in Geographic Information System (GIS). Applying high performance computing to perform geometric intersection on huge amount of spatial data to get real-time results is necessary. Given two input geometries (polygon or polyline) of a candidate pair, we introduce a new two-step geospatial filter that first creates sketches of the geometries and uses it to detect workload and then refines the sketches by the common areas of sketches to decrease the overall computations in the refine phase. We call this filter PolySketch-based CMBR (PSCMBR) filter. We show the application of this filter in …

Toward High Performance Computing Education, Rajendra K. Raj, Carol J. Romanowski, Sherif G. Aly, Brett A. Becker, Juan Chen, Sheikh Ghafoor, Nasser Giacaman, Steven I. Gordon, Cruz Izu, Shahram Rahimi, Michael P. Robson, Neena Thota

Computer Science: Faculty Publications

High Performance Computing (HPC) is the ability to process data and perform complex calculations at extremely high speeds. Current HPC platforms can achieve calculations on the order of quadrillions of calculations per second with quintillions on the horizon. The past three decades witnessed a vast increase in the use of HPC across different scientific, engineering and business communities, for example, sequencing the genome, predicting climate changes, designing modern aerodynamics, or establishing customer preferences. Although HPC has been well incorporated into science curricula such as bioinformatics, the same cannot be said for most computing programs. This working group will explore how …

Performance Modeling And Resource Provisioning For Data-Intensive Applications, Zhongwei Li

Computer Science and Engineering Dissertations

Performance evaluation and resource provisioning are two most critical factors to be considered for designers of distributed systems at modern warehouse data centers. The ever-increasing volumes of data in recent years have pushed many businesses to move their computing tasks to the Cloud, which offers many benefits including the low system management and maintenance costs and better scalability. As a result, most recent prominently emerging workloads are data-intensive, calling for scaling out the workload to a large number of servers for parallel processing. Questions can be asked as what factors impact the system scaling performance, and how to efficiently schedule …

Performance Modeling And Resource Provisioning For Data-Intensive Applications, Zhongwei Li

Computer Science and Engineering Theses

Performance evaluation and resource provisioning are two most critical factors to be considered for designers of distributed systems at modern warehouse data centers. The ever-increasing volumes of data in recent years have pushed many businesses to move their computing tasks to the Cloud, which offers many benefits including the low system management and maintenance costs and better scalability. As a result, most recent prominently emerging workloads are data-intensive, calling for scaling out the workload to a large number of servers for parallel processing. Questions can be asked as what factors impact the system scaling performance, and how to efficiently schedule …

On The Feasibility Of Malware Unpacking With Hardware Performance Counters, Jay Mayank Patel

Computer Science and Engineering Theses

Most of the malware authors use Packers, to compress an executable file and attach a stub, to the file containing the code, to decompress it at runtime, which will turn a known piece of malware into something new, that known-malware scanners can't detect. The researchers are finding ways to unpack and find the original program from such packed binaries. However, the previous study of detection for unpacking in the packed malware using different approach won’t provide many promising results. This research explores a novel approach for the detection of the unpacking process using hardware performance counters. In this approach, the …

Adaptive Parallelism For Coupled, Multithreaded Message-Passing Programs, Samuel K. Gutiérrez

Computer Science ETDs

Hybrid parallel programming models that combine message passing (MP) and shared- memory multithreading (MT) are becoming more popular, especially with applications requiring higher degrees of parallelism and scalability. Consequently, coupled parallel programs, those built via the integration of independently developed and optimized software libraries linked into a single application, increasingly comprise message-passing libraries with differing preferred degrees of threading, resulting in thread-level heterogeneity. Retroactively matching threading levels between independently developed and maintained libraries is difficult, and the challenge is exacerbated because contemporary middleware services provide only static scheduling policies over entire program executions, necessitating suboptimal, over-subscribed or under-subscribed, configurations. In …

Dark Matter Halo Mass Function From Hpc N-Body Simulations, Da Bi, Isaac Shlosman, Emilio Romano-Diaz

Commonwealth Computational Summit

Dark matter (DM) dominates the matter in the Universe. Because of self-gravity, DM collapses and becomes clumpy, building the large-scale hierarchical structures. Baryons assemble within DM potential wells and form galaxies.

Because we can not directly observe DM halos, numerical simulations is the only way one can study their dynamics and other properties. Using N-body simulations, we can obtain the Halo Mass Function (HMF), which provides the abundance of DM halos as a function of their mass. The HMF depends weakly on cosmological redshift and is one of the basic tools in modern cosmology.

We use GIZMO --- a flexible, …

Use Of Hpc To Analyze Changes In Gene Expression During Fruit Fly Spermiogenesis, Sepideh Dadkhah, Douglas A. Harrison, Jeramiah J. Smith

Commonwealth Computational Summit

In the fruit fly, Drosophila melanogaster, JAK/STAT signaling during spermiogenesis is known to play a crucial role in the maintenance of stem cells of the testis. Recent studies in our lab have shown that activation of the JAK/STAT pathway in somatic cyst cells is also required for the later stages of spermiogenesis like individualization.

The main goal of this project is to characterize the events downstream of JAK/STAT signaling in spermiogenesis and more specifically to determine the mechanism by which JAK/STAT activation regulates individualization, a later stage in spermiogenesis where 64 individual spermatids are formed from a 64-interconnected spermatid …

Evaluation Of Deep Learning Frameworks Over Different Hpc Architectures, Shayan Shams, Richard Platania, Kisung Lee, Seung Jong Park

Computer Science Faculty Research & Creative Works

Recent advances in deep learning have enabled researchers across many disciplines to uncover new insights about large datasets. Deep neural networks have shown applicability to image, time-series, textual, and other data, all of which are available in a plethora of research fields. However, their computational complexity and large memory overhead requires advanced software and hardware technologies to train neural networks in a reasonable amount of time. To make this possible, there has been an influx in development of deep learning software that aim to leverage advanced hardware resources. In order to better understand the performance implications of deep learning frameworks …

Programming Models' Support For Heterogeneous Architecture, Wei Wu

Doctoral Dissertations

Accelerator-enhanced computing platforms have drawn a lot of attention due to their massive peak computational capacity. Heterogeneous systems equipped with accelerators such as GPUs have become the most prominent components of High Performance Computing (HPC) systems. Even at the node level the significant heterogeneity of CPU and GPU, i.e. hardware and memory space differences, leads to challenges for fully exploiting such complex architectures. Extending outside the node scope, only escalate such challenges.

Conventional programming models such as data- ow and message passing have been widely adopted in HPC communities. When moving towards heterogeneous systems, the lack of GPU integration causes …

Characterizing And Improving Power And Performance In Hpc Networks, Taylor L. Groves

Computer Science ETDs

Networks are the backbone of modern HPC systems. They serve as a critical piece of infrastructure, tying together applications, analytics, storage and visualization. Despite this importance, we have not fully explored how evolving communication paradigms and network design will impact scientific workloads. As networks expand in the race towards Exascale (1×10^18 floating point operations a second), we need to reexamine this relationship so that the HPC community better understands (1) characteristics and trends in HPC communication; (2) how to best design HPC networks to save power or enhance the performance; (3) how to facilitate scalable, informed, and dynamic decisions within …

Hsp-Wrap: The Design And Evaluation Of Reusable Parallelism For A Subclass Of Data-Intensive Applications, Paul R. Giblock

Masters Theses

There is an increasing gap between the rate at which data is generated by scientific and non-scientific fields and the rate at which data can be processed by available computing resources. In this paper, we introduce the fields of Bioinformatics and Cheminformatics; two fields where big data has become a problem due to continuing advances in the technologies that drives these fields: such as gene sequencing and small ligand exploration. We introduce high performance computing as a means to process this growing base of data in order to facilitate knowledge discovery. We enumerate goals of the project including reusability, efficiency, …

Parallel Processes In Hpx: Designing An Infrastructure For Adaptive Resource Management, Vinay Chandra Amatya

LSU Doctoral Dissertations

Advancement in cutting edge technologies have enabled better energy efficiency as well as scaling computational power for the latest High Performance Computing(HPC) systems. However, complexity, due to hybrid architectures as well as emerging classes of applications, have shown poor computational scalability using conventional execution models. Thus alternative means of computation, that addresses the bottlenecks in computation, is warranted. More precisely, dynamic adaptive resource management feature, both from systems as well as application's perspective, is essential for better computational scalability and efficiency. This research presents and expands the notion of Parallel Processes as a placeholder for procedure definitions, targeted at one …

Data-Intensive Computing For Bioinformatics Using Virtualization Technologies And Hpc Infrastructures, Pengfei Xuan

All Theses

The bioinformatics applications often involve many computational components and massive data sets, which are very difficult to be deployed on a single computing machine. In this thesis, we designed a data-intensive computing platform for bioinformatics applications using virtualization technologies and high performance computing (HPC) infrastructures with the concept of multi-tier architecture, which can seamlessly integrate the web user interface (presentation tier), scientific workflow (logic tier) and computing infrastructure (data/computing tier). We demonstrated our platform on two bioinformatics projects. First, we redesigned and deployed the cotton marker database (CMD) (http://www.cottonmarker.org), a centralized web portal in the cotton research community, using the …

Java Grande Forum Report: Making Java Work For High-End Computing, George K. Thiruvathukal

George K. Thiruvathukal

This document describes the Java Grande Forum and includes its initial deliverables.Theseare reports that convey a succinct set of recommendations from this forum to SunMicrosystems and other purveyors of Java™ technology that will enable GrandeApplications to be developed with the Java programming language.

Designing Reliable High-Performance Storage Systems For Hpc Environments, Lucas Scott Hindman

Boise State University Theses and Dissertations

Advances in processing capability have far outpaced advances in I/O throughput and latency. Distributed file system based storage systems help to address this performance discrepancy in high performance computing (HPC) environments; however, they can be difficult to deploy and challenging to maintain. This thesis explores the design considerations as well as the pitfalls faced when deploying high performance storage systems. It includes best practices in identifying system requirements, techniques for generating I/O profiles of applications, and recommendations for disk subsystem configuration and maintenance based upon a number of recent papers addressing latent sector and unrecoverable read errors.

Efficient Replica-Exchange Across Distributed Production Infrastructure, Abhinav S. Thota

LSU Master's Theses

Replica-Exchange (RE) methods represent a class of algorithms that involve a large number of loosely-coupled ensembles and are used to understand physical phenomena -- ranging from protein folding dynamics to binding affinity calculations. We develop a framework for RE that supports different replica pairing and coordination mechanisms, that can use a wide range of production cyberinfrastructure concurrently. Additionally, our framework uses a flexible pilot-job implementation, which enables effective resource allocation for multiple replicas. We characterize the performance of two different RE algorithms - synchronous and asynchronous - at unprecedented scales on production distributed infrastructure (Teragrid and LONI). The synchronous RE …

A Study Of Improving The Parallel Performance Of Vasp., Matthew Brandon Baker

Electronic Theses and Dissertations

This thesis involves a case study in the use of parallelism to improve the performance of an application for computational research on molecules. The application, VASP, was migrated from a machine with 4 nodes and 16 single-threaded processors to a machine with 60 nodes and 120 dual-threaded processors. When initially migrated, VASP's performance deteriorated after about 17 processing elements (PEs), due to network contention. Subsequent modifications that restrict communication amongst VASP processes, together with additional support for threading, allowed VASP to scale up to 112 PEs, the maximum number that was tested. Other performance-enhancing optimizations that were attempted included replacing …

Java Grande Forum Report: Making Java Work For High-End Computing, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

This document describes the Java Grande Forum and includes its initial deliverables.Theseare reports that convey a succinct set of recommendations from this forum to SunMicrosystems and other purveyors of Java™ technology that will enable GrandeApplications to be developed with the Java programming language.

A Study Of Software Development For High Performance Computing, Manish Parashar, Salim Hariri, Tomasz Haupt, Geoffrey C. Fox

Northeast Parallel Architecture Center

Software development in a High Performance Computing (HPC) environment is non-trivial and requires a thorough understanding of the application and the architecture. The objective of this paper is to study the software development process in a high performance computing environment and to outline the stages typically encountered in this process. Support required at each stage is also highlighted. The modeling of stock option pricing is used as a running example in the study.