Open Access. Powered by Scholars. Published by Universities.®
- Discipline
- Keyword
-
- MDS array codes (2)
- RAID-6 (2)
- Reliability (2)
- Storage system (2)
- Big Data protection (1)
-
- Cluster deduplication (1)
- Data management (1)
- Data routing (1)
- Handprinting (1)
- Large distributed file systems (1)
- Large-scale File Systems (1)
- Load balance (1)
- Metadata (1)
- Namespace Management (1)
- Next generation storage systems and media (1)
- Scalability (1)
- Scalable storage (1)
- Security (1)
- Semantic Sensitivity (1)
- Similarity index (1)
- Storage systems for data intensive computing (1)
- Super- chunk (1)
Articles 1 - 7 of 7
Full-Text Articles in Data Storage Systems
Ultimate Codes: Near-Optimal Mds Array Codes For Raid-6, Zhijie Huang, Hong Jiang, Chong Wang, Ke Zhou, Yuhong Zhao
Ultimate Codes: Near-Optimal Mds Array Codes For Raid-6, Zhijie Huang, Hong Jiang, Chong Wang, Ke Zhou, Yuhong Zhao
School of Computing: Technical Reports
As modern storage systems have grown in size and complexity, RAID-6 is poised to replace RAID-5 as the dominant form of RAID architectures due to its ability to protect against double disk failures. Many excellent erasure codes specially designed for RAID-6 have emerged in recent years. However, all of them have limitations. In this paper, we present a class of near perfect erasure codes for RAID-6, called the Ultimate codes. These codes encode, update and decode either optimally or nearly optimally, regardless of what the code length is. This implies that utilizing these codes we can build highly efficient and …
S-Code: Lowest Density Mds Array Codes For Raid-6, Zhijie Huang, Hong Jiang, Ke Zhou, Yuhong Zhao, Chong Wang
S-Code: Lowest Density Mds Array Codes For Raid-6, Zhijie Huang, Hong Jiang, Ke Zhou, Yuhong Zhao, Chong Wang
School of Computing: Technical Reports
RAID, a storage architecture designed to exploit I/O parallelism and provide data reliability, has been deployed widely in computing systems as a storage building block. In large scale storage systems, in particular, RAID-6 is gradually replacing RAID-5 as the dominant form of disk arrays due to its capability of tolerating concurrent failures of any two disks. MDS (maximum distance separable) array codes are the most popular erasure codes that can be used for implementing RAID-6, since they enable optimal storage efficiency and efficient encoding and decoding algorithms. In this paper, we propose a new class of MDS array codes called …
Providing Flexible File-Level Data Filtering For Big Data Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
Providing Flexible File-Level Data Filtering For Big Data Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
School of Computing: Technical Reports
The enormous amount of big data datasets impose the needs for effective data filtering technique to accelerate the analytics process. We propose a Versatile Searchable File System, VSFS, which provides a transparent, flexible and near real-time file-level data filtering service by searching files directly through the file system. Therefore, big data analytics applications can transparently utilize this filtering service without application modifications. A versatile index scheme is designed to adapt to the exploratory and ad-hoc nature of the big data analytics activities. Moreover, VSFS uses a RAM-based distributed architecture to perform file indexing. The evaluations driven by three real-world analytics …
Vsfs: A Versatile Searchable File System For Hpc Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
Vsfs: A Versatile Searchable File System For Hpc Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson
School of Computing: Technical Reports
Big-data/HPC analytics applications have urgent needs for file-search services to drastically reduce the scale of the input data to accelerate analytics. Unfortunately, the existing solutions either are poorly scalable for large-scale systems, or lack well-integrated interface to allow applications to easily use them. We propose a distributed searchable file system, VSFS, which provide a novel and flexible POSIX-compatible searchable file system namespace that can be seamlessly integrate with any legacy code without modification. Additionally, to provide real-time indexing and searching performance, VSFS uses DRAM-based distributed consistent hashing ring to manages all file-index. The results of our evaluation show that VSFS …
A Scalable Inline Cluster Deduplication Framework For Big Data Protection, Yinjin Fu, Hong Jiang, Nong Xiao
A Scalable Inline Cluster Deduplication Framework For Big Data Protection, Yinjin Fu, Hong Jiang, Nong Xiao
School of Computing: Technical Reports
Cluster deduplication has become a widely deployed technology in data protection services for Big Data to satisfy the requirements of service level agreement (SLA). However, it remains a great challenge for cluster deduplica- tion to strike a sensible tradeoff between the conflicting goals of scalable dedu- plication throughput and high duplicate elimination ratio in cluster systems with low-end individual secondary storage nodes. We propose Σ-Dedupe, a scalable inline cluster deduplication framework, as a middleware deployable in cloud da- ta centers, to meet this challenge by exploiting data similarity and locality to op- timize cluster deduplication in inter-node and intra-node scenarios, …
Rapport: Semantic-Sensitive Namespace Management In Large-Scale File Systems, Yu Hua, Hong Jiang, Yifeng Zhu, Dan Feng
Rapport: Semantic-Sensitive Namespace Management In Large-Scale File Systems, Yu Hua, Hong Jiang, Yifeng Zhu, Dan Feng
School of Computing: Technical Reports
Explosive growth in volume and complexity of data exacerbates the key challenge to effectively and efficiently manage data in a way that fundamentally improves the ease and efficacy of their use. Existing large-scale file systems rely on hierarchically structured namespace that leads to severe performance bottlenecks and renders it impossible to support real-time queries on multi-dimensional attributes. This paper proposes a novel semantic-sensitive scheme, called Rapport, to provide dynamic and adaptive namespace management and support complex queries. The basic idea is to build files’ namespace by utilizing their semantic correlation and exploiting dynamic evolution of attributes to support namespace management. …
Dsfs: Decentralized Security For Large Parallel File Systems, Zhongying Niu, Hong Jiang, Ke Zhou, Dan Feng, Tianming Yang, Dongliang Lei, Anli Chen
Dsfs: Decentralized Security For Large Parallel File Systems, Zhongying Niu, Hong Jiang, Ke Zhou, Dan Feng, Tianming Yang, Dongliang Lei, Anli Chen
School of Computing: Technical Reports
This paper describes DSFS, a decentralized security system for large parallel file system. DSFS stores global access control lists (ACLs) in a centralized decisionmaking server and pushes pre-authorization lists (PALs) into storage devices. Thus DSFS allows users to flexibly set any access control policy for the global ACL or even change the global ACL system without having to upgrade the security code in their storage devices. With pre-authorization lists, DSFS enables a networkattached storage device to immediately authorize I/O, instead of demanding a client to acquire an authorization from a centralized authorization server at a crucial time. The client needs …