Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 2 of 2
Full-Text Articles in Engineering
Concentric Layout, A New Scientific Data Layout For Matrix Data Set In Hadoop File System, Lu Cheng
Concentric Layout, A New Scientific Data Layout For Matrix Data Set In Hadoop File System, Lu Cheng
Electronic Theses and Dissertations
The data generated by scientific simulation, sensor, monitor or optical telescope has increased with dramatic speed. In order to analyze the raw data speed and space efficiently, data preprocess operation is needed to achieve better performance in data analysis phase. Current research shows an increasing tread of adopting MapReduce framework for large scale data processing. However, the data access patterns which generally applied to scientific data set are not supported by current MapReduce framework directly. The gap between the requirement from analytics application and the property of MapReduce framework motivates us to provide support for these data access patterns in …
Researches On Reverse Lookup Problem In Distributed File System, Junyao Zhang
Researches On Reverse Lookup Problem In Distributed File System, Junyao Zhang
Electronic Theses and Dissertations
Recent years have witnessed an increasing demand for super data clusters. The super data clusters have reached the petabyte-scale can consist of thousands or tens of thousands storage nodes at a single site. For this architecture, reliability is becoming a great concern. In order to achieve a high reliability, data recovery and node reconstruction is a must. Although extensive research works have investigated how to sustain high performance and high reliability in case of node failures at large scale, a reverse lookup problem, namely finding the objects list for the failed node remains open. This is especially true for storage …