Access Type

Open Access Dissertation

Date of Award

January 2011

Degree Type

Dissertation

Degree Name

Ph.D.

Department

Computer Science

First Advisor

Lihao Xu

Abstract

Building reliable data storage systems is crucial to any commercial or scientific applications. Modern storage systems are complicated, and they are comprised of many components, from hardware to software. Problems may occur to any component of storage systems and cause data loss. When this kind of failures happens, storage systems cannot continue their data services, which may result in large revenue loss or even catastrophe to enterprises. Therefore, it is critically important to build reliable storage systems to ensure data reliability.

In this dissertation, we propose to employ general erasure codes to build a reliable file system, called HyFS. HyFS is a cluster system, which can aggregate distributed storage servers to provide reliable data service. On client side, HyFS is implemented as a native file system so that applications can transparently run on top of HyFS. On server side, HyFS utilizes multiple distributed storage servers to provide highly reliable data service by employing erasure codes. HyFS is able to offer high throughput for either random or sequential file access, which makes HyFS an attractive choice for primary or backup storage systems.

This dissertation studies five relevant topics of HyFS. Firstly, it presents several algorithms that can perform encoding operation efficiently for XOR-based erasure codes. Secondly, it discusses an efficient decoding algorithm for RAID-6 erasure codes. This algorithm can recover various types of disk failures. Thirdly, it describes an efficient algorithm to detect and correct errors for the STAR code, which further improves a storage system's reliability. Fourthly, it describes efficient implementations for the arithmetic operations of large finite fields. This is to improve a storage system's security. Lastly and most importantly, it presents the design and implementation of HyFS and evaluates the performance of HyFS.

Download

Share

COinS