Data Storage Systems Commons^™

Achieving High Reliability And Efficiency In Maintaining Large-Scale Storage Systems Through Optimal Resource Provisioning And Data Placement, Lipeng Wan

Doctoral Dissertations

With the explosive increase in the amount of data being generated by various applications, large-scale distributed and parallel storage systems have become common data storage solutions and been widely deployed and utilized in both industry and academia. While these high performance storage systems significantly accelerate the data storage and retrieval, they also bring some critical issues in system maintenance and management. In this dissertation, I propose three methodologies to address three of these critical issues.

First, I develop an optimal resource management and spare provisioning model to minimize the impact brought by component failures and ensure a highly operational experience ...

Metadata For Describing Water Models, Mohamed M. Morsy, Jonathan L. Goodall, Christina Bandaragoda, Anthony M. Castronova, Jane Greenberg

Anthony Castronova

Computer models are widely used in hydrology and water resources management. A large variety of models exist, each tailored to address specific challenges related to hydrologic science and water resources management. When scientists and engineers apply one of these models to address a specific question, they must devote significant effort to set up, calibrate, and evaluate that model instance built for some place and time. In many cases, there is a benefit to sharing these computer models and associated datasets with the broader scientific community. Core to model reuse in any context is metadata describing the model. A standardized metadata ...

Scaling Watershed Models: Modern Approaches To Science Computation With Mapreduce, Parallelization, And Cloud Optimization, Jonathan Flaishans, Meridith Fry, Trip Hook, Nelson Thurman, Jim Carleton, Shelly Thawley, Kurt Wolfe, Dirk Young, Tom Purucker

International Congress on Environmental Modelling and Software

Environmental models are products of the computer architecture and software tools available at the time of development. Scientifically sound algorithms may persist in their original state even as system architectures and software development approaches evolve and progress. Dating back to the 1980s, the EPA has developed algorithms to estimate the flux of pesticides from treated fields to neighboring water bodies. Recent development of the EPA’s Spatial Aquatic Model (SAM) has provided an opportunity to redevelop, optimize and modernize this code used for regulatory decisions. Use of profiling has indicated a number of efficiencies that could be gained by updating ...

Ogc Sensor Observation Service: Past, Present, Internet Of The Things, Argyris Samourkasidis, Ioannis N. Athanasiadis

International Congress on Environmental Modelling and Software

Every-day devices are being miniaturized, equipped with transducers and transformed into internet-connected sensor nodes. Above smart-devices, not only can sense their surroundings, but could also store and disseminate their observations. While Internet of the Things (IoT) introduces a new, dynamic, participatory ecosystem of decentralized and collaborative devices, intrinsic heterogeneity hinders its full realization. Diversity of smart-devices and their resulting data formats, demands for interoperable data curation solutions. Open Geospatial Consortium (OGC) has put significant work towards interoperable data interchange. Through Sensor Web Enablement (SWE) and its related service interfaces, and information models, OGC offers syntactic and semantic interoperability. Sensor Observation ...

Chemical Transformation System: Cloud Based Cheminformatic Services To Support Integrated Environmental Modeling, Kurt Wolfe, Nicholas Pope, Rajbir Parmar, Mike Galvin, Caroline Stevens, Eric Weber, Jonathan Flaishans, Tom Purucker

International Congress on Environmental Modelling and Software

Integrated Environmental Modeling (IEM) systems that account for the fate/transport of organics frequently require physicochemical properties as well as transformation products. A myriad of chemical property databases exist but these can be difficult to access and often do not contain the proprietary chemicals that environmental regulators must consider. We are building the Chemical Transformation System (CTS) to facilitate model parameterization and analysis. CTS integrates a number of physicochemical property calculators into the system including EPI Suite, SPARC, TEST and ChemAxon. The calculators are heterogeneous in their scientific methodologies, technology implementations and deployment stacks. CTS also includes a chemical transformation ...

Environmental Models As A Service: Enabling Interoperability Through Restful Endpoints And Api Documentation, Jonathan Flaishans, Mike Galvin, Amber Ignatius, Carmen Kuan, Gerard F. Laniak, Kurt Wolfe, Tom Purucker

International Congress on Environmental Modelling and Software

Achieving interoperability in environmental modeling has evolved as software technology has progressed. The recent rise of cloud computing and proliferation of web services initiated a new stage for creating interoperable systems. Scientific programmers increasingly take advantage of streamlined deployment processes and affordable cloud access to move algorithms and data to the web for discoverability and consumption. In these deployments, environmental models can become available to end users through RESTful web services and consistent application program interfaces (APIs) that consume, manipulate, and store modeling data. RESTful modeling APIs also promote discoverability and guide usability through self-documentation. Embracing the RESTful paradigm allows ...

Design And Implementation Of A Rest Api For The Human Well Being Index (Hwbi), Amber R. Ignatius, Kurt Wolfe, Rajbir Parmar, Jonathan Flaishans, Linda Harwell, Susan Yee, Tom Purucker, Mike Galvin

International Congress on Environmental Modelling and Software

Interoperable software development uses principles of component reuse, systems integration, flexible data transfer, and standardized ontological documentation to promote access, reuse, and integration of code. While interoperability principles are increasingly considered technology standards in software engineering, adoption by the environmental modeling community has been slow. We created an Application Programming Interface (API) for the U.S. Environmental Protection Agency’s Human Well-being Index (HWBI) model based on interoperability principles. The HWBI characterizes economic, social, and environmental services during years 2000-2010. For each county in the U.S., specific metrics (e.g., life expectancy, housing affordability, voter turnout) are used to ...

Cloud Services Integration Platform (Csip) Model And Data Services, Olaf David, W. Lloyd, J. Carlson, M. Arabi, T. Wible

International Congress on Environmental Modelling and Software

The Cloud Services Integration Platform (CSIP) is an open platform tailored to develop and deploy microservices for environmental models with data sources. Its emergence responded to needs within the modelling community for rapid and wide access to such modelling resources. CSIP was designed from the ground up (1) to account for modelling service elasticity and scalability, (2) to leverage contemporary computational approaches, (3) to address interoperability and traceability within operational settings, (4) to allow platform and language agnostic service access and cloud agnostic deployment while providing a simple, OMS3 based non-invasive approach to leverage legacy and new modelling solutions with ...

Improving Component Interoperability And Reusability With The Java Connection Framework (Jcf): Overview And Application To The Ages-W Environmental Model, James C. Ascough Ii, Nathan P. Lighthart, Holm Kipka, Olaf David, Timothy R. Green, Gregory S. Mcmaster

International Congress on Environmental Modelling and Software

Environmental modeling framework (EMF) design goals are multi-dimensional and often include many aspects of general software framework development. Many functional capabilities offered by current EMFs are closely related to interoperability and reuse aspects. For example, an EMF needs to support development productivity and interoperability with other modeling tools, protocols, and programming languages, thereby minimizing overall integration effort. Facilitating data exchange between modules, classes, or components representing parts of a whole model is also an essential EMF feature. In general, EMFs should enable flexible model development rather than enforce complex and rigid development concepts. Additionally, full-featured EMFs should enable module connectivity ...

From Raw Material To Biofuel Demand: The Agraf Model, Coline Assaiante, Pierre-Alain Jayet, Frédérick Lantz

International Congress on Environmental Modelling and Software

The search for energetic independence, for boosting local economies and for reducing polluting emissions, has supported bioenergy’s development and biofuels in particular. Mandatory biofuels incorporation rates have been recently established in several countries, including the European Union, in order to promote the use of renewable energetic resources in spite of several criticisms toward biofuels relating to their impacts on food security, the environment and the conflicts associated with resource allocations. To study existing concerns on the growth, the profitability, the limiting factors biofuels, as well as the impacts of current policies in force, a highly parameterized operational modelling tool ...

Replicability Of A Modelling Solution Using Newage-Jgrass, Marialaura Bancheri, Giuseppe Formetta, Francesco Serafin, Riccardo Rigon, Timothy R. Green, Olaf David

International Congress on Environmental Modelling and Software

NewAge-JGrass is a semi-distributed hydrological modelling system. It is based on the object-oriented modelling framework (OMS version 3), on the JGrasstools and on the Geotools. To make it more effective, it is interoperable with many other tools such as R-project, gvSIG GIS, GitHub and other collaborative sharing tools, such as Authorea.

OMS3 allows to create science simulation components, which are implemented with a standard, well-defined purpose and interface. The encapsulation of the source code in components make them easier to be inspected and its copyleft license allows the easy access to the code, its use and its improvement. The components ...

One Dimensional Riverine Water Quality Modeling System (Hec-Ras/Nsm), Billy E. Johnson, Zhonglong Zhang

International Congress on Environmental Modelling and Software

The transport of nutrients from watersheds to aquatic resources (streams and rivers, lakes and reservoirs, and coastal zones) directly influences their environmental quality and ecosystem. While this is a natural process, excessive inputs from anthropogenic sources (e.g., intensive agriculture) can exceed the assimilative capacity and resilience of aquatic ecosystems. The result is accelerated rates of eutrophication and destabilized ecological communities susceptible to the establishment of invasive species or to changes in food web structure, loss of valuable habitat, threats to human health, and reduced economic and societal benefits. As part of the Ecosystem Management and Restoration Research Program (EMRRP ...

Time Series Analyst: Interactive Online Visualization Of Standards Based Environmental Time Series Data, Jeffery S. Horsburgh, Amber Spackman Jones, Maurier Ramírez, Juan Caraballo

International Congress on Environmental Modelling and Software

Visualization is a common need of many researchers, organizations, and projects that collect and use environmental sensor data. Web-based tools can provide screening-level visualization and data analysis functionality for users with a range of technical expertise. However, it is difficult to integrate data from multiple sources due to syntactic and semantic differences in data formats and different data delivery mechanisms that are not always interoperable. We developed an open-source time series visualization tool called the Time Series Analyst (TSA) that integrates data from multiple sources using web service interfaces and a standardized data encoding format. The TSA provides multiple types ...

Web Based Analysis Of Hydrological Time Series In R Using Web Processing Services, Jiří Kadlec, Daniel P. Ames, Matthew Bayles, Martin Seul, Rick Hooper, Brian Cummings

International Congress on Environmental Modelling and Software

An increasing quantity of observed and modeled hydrological time series data are becoming available through web based catalogs and data servers. The Consortium of Universities for the Advancement of Hydrologic Sciences, Inc. (CUAHSI) Water Data Center makes much of this data available globally via its new http://data.cuahsi.org web portal. A computationally distributed system was needed to facilitate web based visualization and analysis of data discovered via this portal. This presentation will introduce the design and development of a web-based software framework including an application programmer interface and architecture to share time series analysis R scripts online. Using ...

Innovation For Consistent Integrated Modeling Across Scales, Bert Jagers

International Congress on Environmental Modelling and Software

In the context of environmental management policies and implementation plans, we are often faced by complex questions about the environment and human interaction. In order to be able to identify solution pathways forward, we need to understand the dynamics of the natural system as a whole and the human role therein. Our understanding of the system is reflected by the components of our numerical models: from highly specific detailed models to more integrated lumped models. We use models across a wide range of scales. Since policies and implementations interact across these scales and the expertise of relevant authorities involved varies ...

Interoperability Between The Basic Modeling Interface (Bmi) And The Open Modeling Interface (Openmi): A Step Toward Building The Earth System Bridge For Modeling Framework Interoperability, Jonathan L. Goodall, Scott D. Peckham

International Congress on Environmental Modelling and Software

Interoperability among environmental modeling frameworks is important to allow for the exchange of software components for modeling, analysis, and data access between these systems. Modeling frameworks share the concept of encapsulating models as software components or services within a larger modeling framework where the components have defined interfaces and data exchange formats. However, the exact definition of these interfaces and data exchange formats differs across modeling frameworks. The Earth System Bridge project, funded by the US National Science Foundation (NSF) under their EarthCube program, seeks to address this challenge by adopting a core model component that can be wrapped using ...

From Openmi To Hydrocouple: Advancing Openmi To Support Experimental Simulations And Standard Geospatial Datasets, Caleb A. Buahin, Jeffery S. Horsburgh

International Congress on Environmental Modelling and Software

HydroCouple is a cross-platform, component-based modeling interface definition that largely follows the Open Modeling Interface 2.0 (OpenMI) specification. HydroCouple provides advancements to better facilitate those experimental model investigations that fall into the so-called “embarrassingly parallel” class of simulations, including uncertainty assessment, ensemble forecasting, and optimization simulations. Additionally, HydroCouple explicitly incorporates low level interface definitions for multi- dimensional datasets and geospatial data formats including the Open Geospatial Consortium’s Simple Feature Access specification, raster datasets, and meshes that are widely used in the earth systems and environmental modeling field. In this paper, we describe these and other advances provided by ...

Artificial Intelligence Techniques To Enhance Actors’ Decision Strategies In Socio-­Ecological Agent-­ Based Models, Shaheen A. Abdulkareem, Ellen-­Wien Augustijn-­Beckers, Yaseen T. Mustafa, Tatiana Filatova

International Congress on Environmental Modelling and Software

Agent-­Based Models (ABMs) are indispensable to studying the aggregated impacts of individual actions of heterogeneous interacting adaptive agents. Concurrently, artificial intelligence has been employed for decades to simulate autonomous actions of individual entities that react, learn and exchange information with an environment and one another. There are obvious synergies between the two computational approaches. For example, artificial intelligence is often used to enhance agents’ behaviour in ABMs. Artificial intelligence learning algorithms (AILAs) allow for a richer agents’ architecture for operationalization of more realistic learning decisions beyond a simplistic treatment of agents’ cognitive and sensory capacities. Firstly, we review recent ...

Gis Mash-Ups In The Cloud: Visualizing Online Data Using Hydroshare And Tethys Platform, Daniel P. Ames, Shawn Crawley, Zhiyu Li

International Congress on Environmental Modelling and Software

HydroShare is a web based data management system for climate and water data that is includes a web-services application programmer interface (API) to allow third party programmers to access and use its open data resources. To provide end users with data analysis and visualization tools for HydroShare resources, we have implemented an online web app development system called Tethys Platform and have developed several apps that utilize the HydroShare API. Tethys Platform is an open source software package based on the Django framework and Python programming language. The Tethys software development kit allows users to create web apps for visualizing ...

Analysing The Error Potential Of Administrative Data In Determining The Sinuosity Of Small Rivers, Pierre Karrasch, Sebastian Hunger

International Congress on Environmental Modelling and Software

According to the European Water Framework Directive the member states are committed to prove the good ecological status of all water bodies. These requirements are verified by a regular monitoring. Especially the use of remote sensing data and methods of applied geoinformatics and statistics can be recommended for the monitoring of several hydromorphological parameters. At the same time for some single parameters basic geodata provided by local authorities are available (e.g. river centrelines). In this study the quality of these basic geodata is compared with self-extracted results using different remote sensing data sources (CIR-orthophoto, digital terrain model). The results ...