Systems Architecture Commons^™

Big Data Investment And Knowledge Integration In Academic Libraries, Saher Manaseer, Afnan R. Alawneh, Dua Asoudi

Copyright, Fair Use, Scholarly Communication, etc.

Recently, big data investment has become important for organizations, especially with the fast growth of data following the huge expansion in the usage of social media applications, and websites. Many organizations depend on extracting and reaching the needed reports and statistics. As the investments on big data and its storage have become major challenges for organizations, many technologies and methods have been developed to tackle those challenges.

One of such technologies is Hadoop, a framework that is used to divide big data into packages and distribute those packages through nodes to be processed, consuming less cost than the traditional storage …

Archive - A Data Management Program, James H. Devilbiss, C. Steven Whisnant, Yasmeen Shorish

Department of Physics and Astronomy - Faculty Scholarship

To meet funding agency requirements, a portable data management solution is presented for small research groups. The database created is simple, searchable, robust, and can reside across multiple hard drives. Employing a standard metadata schema for all data, the database ensures a high level of standardization, findability, and organization. The software is written in Perl, runs on UNIX, and presents a web-based user interface. It uses a fast, portable log-in scheme, making it easy to export to other locations. As research continues to move towards more open data sharing and reproducibility, this database solution is agile enough to accommodate external …

Vsfs: A Versatile Searchable File System For Hpc Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson

CSE Technical Reports

Big-data/HPC analytics applications have urgent needs for file-search services to drastically reduce the scale of the input data to accelerate analytics. Unfortunately, the existing solutions either are poorly scalable for large-scale systems, or lack well-integrated interface to allow applications to easily use them. We propose a distributed searchable file system, VSFS, which provide a novel and flexible POSIX-compatible searchable file system namespace that can be seamlessly integrate with any legacy code without modification. Additionally, to provide real-time indexing and searching performance, VSFS uses DRAM-based distributed consistent hashing ring to manages all file-index. The results of our evaluation show that VSFS …