Because no central authority can arrange how data are distributed among and retrieved from the whole storage system, a storage server independently manages the stored data without global information of the networked stor-age system. Although the data manstor-agement without a central authority is much more complicated, the decentralized architecture has many advantages in practice. A decentralized architecture for storage systems offers a good
scalability since any storage server can join or leave without the control of a central authority. The architecture can bear a global-size system scale and the resulting massive storage space. Peer-to-peer networks are major concrete examples for the decentralized architecture.
In this section, We describes several decentralized networked storage servers that use different robust storage technologies.
2.2.1 Mirrors and Replicas
When there is no central authority, a datum can be flooded into the storage system and each (available) storage server stores a copy of it. PAST [24]
leaves the choice of a replication factor, i.e. the number of replicas, to the data owner. Farsite [10] is a reliable file system that uses random replication to support long-term data persistence. Glacier [25] is a distributed storage system that uses massive replication to provide data robustness across large-scale correlated failures. Many strategies aim to arrange where a replica is stored such that this replica can be found when needed. Carbonite [26] is a replication algorithm for keeping data durable at a low cost. It contributes to distributed storage prototypes such as Antiquity [27], OverCite [28], and UsenetDHT [29].
Consider a peer-to-peer network as a special case of a networked storage system. The replica placement and the quality of availability are a huge research area in peer-to-peer networks. For example, a repair mechanism handles how replicas should be re-distributed when a peer leaves the network.
In addition to the static control over the number of available replicas,
proactive replication [30, 31] provides a better guarantee for data durabil-ity. The replication algorithm in [30] produces replicas periodically. The replication algorithm in [31] generates replicas by predicting the machine availability.
2.2.2 Erasure Codes
Similar as the case in the centralized architecture, applying an erasure code provides a lower storage overhead while the resulting system retains the data robustness. Technologies such as fountain codes, decentralized erasure codes, or random linear codes are proposed for the decentralized storage systems [32, 33, 34, 35, 36, 37, 38].
XOR Operation based Erasure Codes
Fountain codes enable a robust storage technique. Luby transform (LT) codes and Raptor codes [39, 40] are two classes of fountain codes. In a fountain code, the message symbols and codeword symbols can be modeled as vertices in a random bipartite graph. The degree of each vertex is random over some probabilistic distribution. Different distributions define different codes.
Figure 2.11 illustrates an overview of the fountain codes. In the LT codes, each codeword symbol is the result of exclusive-or over d message symbols, where the value d is random according to an Ideal Soliton distribution and a robust Soliton distribution. The Raptor code is a class of the fountain code with linearly encoding and decoding complexity. It uses a modification of the Ideal Soliton distribution for the degree d. The distributed storage systems
Figure 2.11: An abstract overview of a fountain code.
The data A, B, C and D are randomly distributed according to some probabilistic distribution. Different codes define different distributions.
using LT codes [35] and the system using Raptor codes [37, 38] are proposed in wireless sensor networks.
Algebraic Operation based Erasure Codes
A random linear code is a linear code with a random generator matrix. Each storage server can randomly determine a column of the generator matrix.
The result in [34] shows that a sub-square matrix of the generator matrix is invertible with an overwhelming probability. That is, the probability that a user can successfully retrieve his data with an overwhelming probability.
Figure 2.12 shows an example of a small storage system that uses a random linear code. There are 6 storage servers SS1, SS2, ..., and SS6 in the system.
After receiving some message symbols, a storage server randomly picks co-efficients and linearly combines the received message symbols. The storage server SS1 stores the codeword symbol C1 and the coefficients x1a, x1b, x1c and x1d, where x1d = 0.
The number of codeword symbols that a datum is contributed could in-fluence the probability of a successful retrieval. Therefore, a decentralized erasure code [34] is proposed for certain settings. The system is summarized
Figure 2.12: A networked storage system using a random linear code.
Each storage server selects a random coefficient for each received message. The linear combination is performed by the storage server. Each column of the gener-ator matrix and each codeword symbol is determined and computed by a storage server. No central authority is required in the encoding process.
in Figure 2.13. In this approach, to store k data, a user makes v copies for each datum and distributes them to randomly selected storage servers in the storage system. After receiving those data, each storage server encodes those data and stores the result. To retrieve k data, the user randomly queries k storage servers for the encoding results and tries to decode those k data. It has been shown that for n storage servers and k messages, if n = ak and v = bk ln k where b > 5a, the probability of successful retrieval is at least 1 − k/p − o(1), where p is the size of the used group [34].
2.2.3 Hybrid Strategy
The hybrid strategy is a mixed method of the replication and an erasure code.
It trades off the advantages and disadvantages between those two approaches.
Many well-known and newly decentralized networked storage systems, such as OceanStore [41] and Total Recall [42], take the hybrid strategy. The
Figure 2.13: A networked storage system using a decentralized erasure code.
The settings for the parameters n, k, and v and the random process of data distri-bution determine the probability of a successful data retrieval.
Figure 2.14: A networked storage system that uses a hybrid strategy.
The storage system simultaneously stores the data and the resulting symbols of the erasure coding. The number of the replicas of the data and the number of symbols of the rasure code influence the data availability of the storage system.
storage servers are divided into primary storage servers that keep the replicas and the secondary storage servers that store the encoding results.
The ratio of two kinds of storage servers may influence the data availabil-ity. Some research [43, 44, 45] are conducted to find an approximated-optimal ratio for providing a practically long-term data availability. Figure 2.14 gives a layout for an erasure coding with primary copy. The experimen-tal results [45] are produced by running experiments on the real traces of large-scale distributed storage systems (e.g. Farsite [10] and Overnet [46]).