Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 3 of 3
Full-Text Articles in Systems Architecture
Providing Flexible File-Level Data Filtering For Big Data Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
Providing Flexible File-Level Data Filtering For Big Data Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
CSE Technical Reports
The enormous amount of big data datasets impose the needs for effective data filtering technique to accelerate the analytics process. We propose a Versatile Searchable File System, VSFS, which provides a transparent, flexible and near real-time file-level data filtering service by searching files directly through the file system. Therefore, big data analytics applications can transparently utilize this filtering service without application modifications. A versatile index scheme is designed to adapt to the exploratory and ad-hoc nature of the big data analytics activities. Moreover, VSFS uses a RAM-based distributed architecture to perform file indexing. The evaluations driven by three real-world analytics …
Vsfs: A Versatile Searchable File System For Hpc Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
Vsfs: A Versatile Searchable File System For Hpc Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
CSE Technical Reports
Big-data/HPC analytics applications have urgent needs for file-search services to drastically reduce the scale of the input data to accelerate analytics. Unfortunately, the existing solutions either are poorly scalable for large-scale systems, or lack well-integrated interface to allow applications to easily use them. We propose a distributed searchable file system, VSFS, which provide a novel and flexible POSIX-compatible searchable file system namespace that can be seamlessly integrate with any legacy code without modification. Additionally, to provide real-time indexing and searching performance, VSFS uses DRAM-based distributed consistent hashing ring to manages all file-index. The results of our evaluation show that VSFS …
Dsfs: Decentralized Security For Large Parallel File Systems, Zhongying Niu, Hong Jiang, Ke Zhou, Dan Feng, Tianming Yang, Dongliang Lei, Anli Chen
Dsfs: Decentralized Security For Large Parallel File Systems, Zhongying Niu, Hong Jiang, Ke Zhou, Dan Feng, Tianming Yang, Dongliang Lei, Anli Chen
CSE Technical Reports
This paper describes DSFS, a decentralized security system for large parallel file system. DSFS stores global access control lists (ACLs) in a centralized decisionmaking server and pushes pre-authorization lists (PALs) into storage devices. Thus DSFS allows users to flexibly set any access control policy for the global ACL or even change the global ACL system without having to upgrade the security code in their storage devices. With pre-authorization lists, DSFS enables a networkattached storage device to immediately authorize I/O, instead of demanding a client to acquire an authorization from a centralized authorization server at a crucial time. The client needs …