Data Storage Systems Commons^™

Closing The Data-Decisions Loop: Deploying Artificial Intelligence For Dynamic Resource Management, Pradeep Varakantham

Asian Management Insights

Improving predictions and allocations to determine the optimal matching of demand and supply in a dynamic, uncertain future.

Big Data Investment And Knowledge Integration In Academic Libraries, Saher Manaseer, Afnan R. Alawneh, Dua Asoudi

Copyright, Fair Use, Scholarly Communication, etc.

Recently, big data investment has become important for organizations, especially with the fast growth of data following the huge expansion in the usage of social media applications, and websites. Many organizations depend on extracting and reaching the needed reports and statistics. As the investments on big data and its storage have become major challenges for organizations, many technologies and methods have been developed to tackle those challenges.

One of such technologies is Hadoop, a framework that is used to divide big data into packages and distribute those packages through nodes to be processed, consuming less cost than the traditional storage …

A Centralized Management System For Raster Data, Felix Linde, Ralf Wieland, Karin Groth

International Congress on Environmental Modelling and Software

We present a concept for a system to manage and distribute geo-referenced raster data from multiple possible sources on the institute level. The core concept is to centralize all import routines for data from different sources and formats, store the information in a common data format and distribute the needed information for research purposes upon request. By using hdf5 files, it can be assured that geo-spatial data is always kept in close connection with the corresponding meta-data. Furthermore this approach builds a technical basis for enhancing data fusion methods by allowing the use of common analysis routines, regardless of the …

Practical Guidance For Integrating Data Management Into Long-Term Ecological Monitoring Projects, Robert D. Sutter, Susan Wainscott, John R. Boetsch, Craig Palmer, David J. Rugg

Library Faculty Research

Long-term monitoring and research projects are essential to understand ecological change and the effectiveness of management activities. An inherent characteristic of long-term projects is the need for consistent data collection over time, requiring rigorous attention to data management and quality assurance. Recent papers have provided broad recommendations for data management; however, practitioners need more detailed guidance and examples. We present general yet detailed guidance for the development of comprehensive, concise, and effective data management for monitoring projects. The guidance is presented as a graded approach, matching the scale of data management to the needs of the organization and the complexity …

Presentation On Evaluating The Creation And Preservation Challenges Of Photogrammetry-Based 3d Models, Michael J. Bennett

UConn Library Presentations

No abstract provided.

Evaluating The Creation And Preservation Challenges Of Photogrammetry-Based 3d Models, Michael J. Bennett

Published Works

Though the roots of photogrammetry can be traced back to photography’s earliest days, only recent advances in both digital technology and software applications have put the possibilities of 3D modeling from 2D source images in the hands of the greater cultural heritage community. The possibilities of such 3D digital rendering are many. With these possibilities come unique digital preservation challenges. This study explores basic close-range photogrammetry as applied to sample archival objects. Additionally, the latest BagIt and ZIP-based bundling formats along with repository-based solutions are also surveyed as potential 3D data management and archiving aggregators for resulting 3D models.

Foss Big Data Storage Solution, Gary L. Jaffe

STAR Program Research Presentations

Utilizing the AERO Institute as an IT test bed or “sandbox”, a small-agile development team will design, build, and test a data management storage system to support post processing of archived and in-flight data collected with the Piccolo flight control system and Compact Fiber Optic Sensing System (C-FOSS). Both systems are integrated on the APV3 aircraft, a small remote-operated vehicle. Due to the amount of data collected from C-FOSS, a system will be designed to sort and organize large data sets. An open-source database will be explored as a viable solution to manage large data loads and provide multi-cluster system …

Automating Data Management And Sharing Within A Large-Scale, Heterogeneous Sensor Network, Jeffery S. Horsburgh, Amber Spackman Jones, Stephanie Reeder

International Congress on Environmental Modelling and Software

Hydrology researchers are collecting data using in situ sensors at high frequencies, for extended durations, and with spatial distributions that require infrastructure for data storage, management, and sharing. Managing streaming sensor data is challenging, especially in large networks with large numbers of sites and sensors. The availability and utility of these data in addressing scientific questions related to water availability, water quality, and natural disasters relies on effective cyberinfrastructure that facilitates transformation of raw sensor data into usable data product. It also depends on the ability of researchers to share and access the data in usable formats. In this paper …

Web Service And Plug-In Architecture For Flexibility And Openness Of Environmental Data Sharing Platforms, Stephen Knox, Philipp Meier, Julien Harou

International Congress on Environmental Modelling and Software

The sharing, comparison and maintenance of computer model input and result data for real-world projects is a challenge. In collaborative research projects, particularly when collaboration occurs between multiple institutions in different locations, the complications are greater still. The co-development of water resource network simulation and optimization models is one field that faces this challenge. There is a need for ways to allow multiple users to build and apply models collaboratively. This paper argues that a web service architecture is an appropriate approach for multi-institution maintenance of resource network models. We describe the design of a prototype system that allows developers …

Data Management In Cloud Environments: Nosql And Newsql Data Stores, Katarina Grolinger, Wilson A. Higashino, Abhinav Tiwari, Miriam Am Capretz

Electrical and Computer Engineering Publications

: Advances in Web technology and the proliferation of mobile devices and sensors connected to the Internet have resulted in immense processing and storage requirements. Cloud computing has emerged as a paradigm that promises to meet these requirements. This work focuses on the storage aspect of cloud computing, specifically on data management in cloud environments. Traditional relational databases were designed in a different hardware and software era and are facing challenges in meeting the performance and scale requirements of Big Data. NoSQL and NewSQL data stores present themselves as alternatives that can handle huge volume of data. Because of the …

Vsfs: A Versatile Searchable File System For Hpc Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson

School of Computing: Technical Reports

Big-data/HPC analytics applications have urgent needs for file-search services to drastically reduce the scale of the input data to accelerate analytics. Unfortunately, the existing solutions either are poorly scalable for large-scale systems, or lack well-integrated interface to allow applications to easily use them. We propose a distributed searchable file system, VSFS, which provide a novel and flexible POSIX-compatible searchable file system namespace that can be seamlessly integrate with any legacy code without modification. Additionally, to provide real-time indexing and searching performance, VSFS uses DRAM-based distributed consistent hashing ring to manages all file-index. The results of our evaluation show that VSFS …

Sheep Updates 2005 - Part 2, Peter Tozer, Janette Drew, K. E. Kemper, M. L. Hebart, F. D. Brien, K. S. Jaensch, R. J. Grimson, D. H. Smith, A. M. M. Ramsay, J. S. Richards, K. D. Atkins, T. Thompson, W. K. Murray, J. R. L. Hall, Chris Carter, Peter Tozer, Mark Hyder, John Young, B. A. Mcgregor, Patricia Harper

Sheep Updates

This session covers seven papers from different authors: CONCURRENT SESSIONS - STRATEGIC MANAGEMENT 1.Finishing Pastoral Lambs, Peter Tozer, Patricia Harper, Janette Drew, Department of Agriculture Western Australia 2. Coating Improves Wool Quality under Mixed Farming Conditions, KE Kemper, ML Hebart, FD Brien, KS Grimson, DH Smith AMM Ramsay, South Australian Research and Development Institute 3. J. S. Richards, K.D. Atkins, T. Thompson, W. K. Murray, Australian Sheep Industry Co-operative Research Centre and NSW Department of Primary Industries, Orange Agricultural Institute, Forest Rd. Orange 4. Strategic Risk Management in the Sheep Industry, J.R.L. Hall, ICON Agriculture (JRL Hall & Co) 5. …