Programming Languages and Compilers Commons^™

Evaluation Of Distributed Programming Models And Extensions To Task-Based Runtime Systems, Yu Pei

Doctoral Dissertations

High Performance Computing (HPC) has always been a key foundation for scientific simulation and discovery. And more recently, deep learning models' training have further accelerated the demand of computational power and lower precision arithmetic. In this era following the end of Dennard's Scaling and when Moore's Law seemingly still holds true to a lesser extent, it is not a coincidence that HPC systems are equipped with multi-cores CPUs and a variety of hardware accelerators that are all massively parallel. Coupling this with interconnect networks' speed improvements lagging behind those of computational power increases, the current state of HPC systems is …

Languages And Compilers For Writing Efficient High-Performance Computing Applications, Abhinav Jangda

Doctoral Dissertations

Many everyday applications, such as web search, speech recognition, and weather prediction, are executed on high-performance systems containing thousands of Central Processing Units (CPUs) and Graphics Processing Units (GPUs). These applications can be written in either low-level programming languages, such as NVIDIA CUDA, or domain specific languages, like Halide for image processing and PyTorch for machine learning programs. Despite the popularity of these languages, there are several challenges that programmers face when developing efficient high-performance computing applications. First, since every hardware support a different low-level programming model, to utilize new hardware programmers need to rewrite their applications in another programming …

Task-Based Runtime Optimizations Towards High Performance Computing Applications, Qinglei Cao

Doctoral Dissertations

The last decades have witnessed a rapid improvement of computational capabilities in high-performance computing (HPC) platforms thanks to hardware technology scaling. HPC architectures benefit from mainstream advances on the hardware with many-core systems, deep hierarchical memory subsystem, non-uniform memory access, and an ever-increasing gap between computational power and memory bandwidth. This has necessitated continuous adaptations across the software stack to maintain high hardware utilization. In this HPC landscape of potentially million-way parallelism, task-based programming models associated with dynamic runtime systems are becoming more popular, which fosters developers’ productivity at extreme scale by abstracting the underlying hardware complexity.

In this context, …

Using Applications To Guide Data Management For Emerging Memory Technologies, Timothy C. Effler

Doctoral Dissertations

A number of promising new memory technologies, such as non-volatile, storage-class memories and high-bandwidth, on-chip RAMs, are emerging. Since each of these new technologies present tradeoffs distinct from conventional DRAMs, many high performance and scientific computing systems have begun to include multiple tiers of memory storage, each with their own type of devices. To efficiently utilize the available hardware, such systems will need to alter their data management strategies to consider the performance and capabilities provided by each tier. This work aims to understand and increase the effectiveness of application data management for emerging complex memory systems. A key realization …

Righting Web Development, John Vilk

Doctoral Dissertations

The web browser is the most important application runtime today, encompassing all types of applications on practically every Internet-connected device. Browsers power complete office suites, media players, games, and augmented and virtual reality experiences, and they integrate with cameras, microphones, GPSes, and other sensors available on computing devices. Many apparently native mobile and desktop applications are secretly hybrid apps that contain a mix of native and browser code. History has shown that when new devices, sensors, and experiences appear on the market, the browser will evolve to support them. Despite the browser's importance, developing web applications is exceedingly difficult. Web …

Run-Time Program Phase Detection And Prediction, Meng-Chieh Chiu

Doctoral Dissertations

It is well-known that programs tend to have multiple phases in their execution. Because phases have impact on micro-architectural features such as caches and branch predictors, they are relevant to program performance (Xian et al., 2007; Roh et al., 2009; Gu and Verbrugge, 2008) and energy consumption. They are also relevant to detecting whether a program is executing as expected or is encountering unusual or exceptional conditions, a software engineering and program monitoring concern (Peleg and Mendelson, 2007; Singer and Kirkham, 2008; Pirzadeh et al., 2011; Benomar et al., 2014). We present methods for real-time phase change detection and phase …

Spreadsheet Tools For Data Analysts, Daniel W. Barowy

Doctoral Dissertations

Spreadsheets are a natural fit for data analysis, combining a simple data storage and presentation layer with a programming language and basic debugging tools. Because spreadsheets are accessible and flexible, they are used by both novices and experts. Consequently, spreadsheets are hugely popular, with more than 750 million copies of Microsoft Excel installed worldwide. This popularity means that spreadsheets are the most popular programming language on the planet and the de facto tool for data analysis. Nevertheless, spreadsheets do not address a number of important tasks in a typical analyst's pipeline, and their design frequently complicates them. This thesis describes …

Programming Models' Support For Heterogeneous Architecture, Wei Wu

Doctoral Dissertations

Accelerator-enhanced computing platforms have drawn a lot of attention due to their massive peak computational capacity. Heterogeneous systems equipped with accelerators such as GPUs have become the most prominent components of High Performance Computing (HPC) systems. Even at the node level the significant heterogeneity of CPU and GPU, i.e. hardware and memory space differences, leads to challenges for fully exploiting such complex architectures. Extending outside the node scope, only escalate such challenges.

Conventional programming models such as data- ow and message passing have been widely adopted in HPC communities. When moving towards heterogeneous systems, the lack of GPU integration causes …

Combining Static And Dynamic Analysis For Bug Detection And Program Understanding, Kaituo Li

Doctoral Dissertations

This work proposes new combinations of static and dynamic analysis for bug detection and program understanding. There are 3 related but largely independent directions: a) In the area of dynamic invariant inference, we improve the consistency of dynamically discovered invariants by taking into account second-order constraints that encode knowledge about
invariants; the second-order constraints are either supplied by the programmer or vetted by the programmer (among candidate constraints suggested automatically); b) In the area of testing dataflow (esp. map-reduce) programs, our tool, SEDGE, achieves higher testing coverage by leveraging existing
input data and generalizing them using a symbolic reasoning engine …

Effective Performance Analysis And Debugging, Charles M. Curtsinger

Doctoral Dissertations

Performance is once again a first-class concern. Developers can no longer wait for the next generation of processors to automatically "optimize" their software. Unfortunately, existing techniques for performance analysis and debugging cannot cope with complex modern hardware, concurrent software, or latency-sensitive software services. While processor speeds have remained constant, increasing transistor counts have allowed architects to increase processor complexity. This complexity often improves performance, but the benefits can be brittle; small changes to a program’s code, inputs, or execution environment can dramatically change performance, resulting in unpredictable performance in deployed software and complicating performance evaluation and debugging. Developers seeking to …

Using Formal Methods To Verify Transactional Abstract Concurrency Control, Trek S. Palmer

Doctoral Dissertations

Concurrent application design and implementation is more important than ever in today's multi-core processor world. Transactional Memory (TM) Concurrent application design and implementation is more important than ever in today's multi-core processor world. Transactional Memory (TM). Each has its own particular advantages and disadvantages. However, these techniques each need some extra information to `glue' the non-transactional operation into a transactional context. At the most general level, non-transactional code must be decorated in such a way that the TM run-time can determine how those non-transactional operations commute with one another, and how to `undo' the non-transactional operations in case the run-time …

Subtyping With Generics: A Unified Approach, John G. Altidor

Doctoral Dissertations

Reusable software increases programmers' productivity and reduces repetitive code and software bugs. Variance is a key programming language mechanism for writing reusable software. Variance is concerned with the interplay of parametric polymorphism (i.e., templates, generics) and subtype (inclusion) polymorphism. Parametric polymorphism enables programmers to write abstract types and is known to enhance the readability, maintainability, and reliability of programs. Subtyping promotes software reuse by allowing code to be applied to a larger set of terms. Integrating parametric and subtype polymorphism while maintaining type safety is a difficult problem. Existing variance mechanisms enable greater subtyping between parametric types, but they suffer …

Reliable And Efficient Multithreading, Tongping Liu

Doctoral Dissertations

The advent of multicore architecture has increased the demand for multithreaded programs. It is notoriously far more challenging to write parallel programs correctly and efficiently than sequential ones because of the wide range of concurrency errors and performance problems. In this thesis, I developed a series of runtime systems and tools to combat concurrency errors and performance problems of multithreaded programs. The first system, Dthreads, automatically ensures determinism for unmodified C/C++ applications using the pthreads library without requiring programmer intervention and hardware support. Dthreads greatly simplifies the understanding and debugging of multithreaded programs. Dthreads often matches or even exceeds the …

Parallel For Loops On Heterogeneous Resources, Frederick Edward Weber

Doctoral Dissertations

In recent years, Graphics Processing Units (GPUs) have piqued the interest of researchers in scientific computing. Their immense floating point throughput and massive parallelism make them ideal for not just graphical applications, but many general algorithms as well. Load balancing applications and taking advantage of all computational resources in a machine is a difficult challenge, especially when the resources are heterogeneous. This dissertation presents the clUtil library, which vastly simplifies developing OpenCL applications for heterogeneous systems. The core focus of this dissertation lies in clUtil's ParallelFor construct and our novel PINA scheduler which can efficiently load balance work onto multiple …