AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
Duplicacy erasure coding1/13/2024 Given that I wanted to get my large Linux system backup underway asap, I turned on EC upon storage creation in B2 before I fully understood what it was, as I gathered enough to understand it would be more difficult to add later. My question is, what are the implications of enabling it on Online Storage? What are the ideal situations in which you would enable it? For example, say you had a Local NAS, would the NAS be aware of Duplicacy’s EC setting? And would the Data:Parity ratio set on Duplicacy need to match the number of drives in your NAS? I have read the Backblaze Blog regarding EC, and watched some YouTube vids on it too, and I think I have wrapped my head around it now. I’ve been out of the loop with Duplicacy for quite a while, so Erasure Coding was a new feature for me to get my head around upon returning to it. I’m currently in the process of doing a complete system backup of my linux system to Backblaze B2. Due to the overhead of managing the number of fragments associated with an erasure coded copy, do not use Erasure Coding profiles for objects smaller than one megabyte.Hi. A large object that is erasure coded and distributed across sites will take longer to retrieve than an object that is replicated and available locally (the same site to which the client connects). At the same time, to protect replicated object data from a site loss, a StorageGRID Webscale deployment requires a minimum of two sites.ĭepending on the configuration of storage pools, it may take longer to retrieve an erasure coded copy than a replicated copy. For example, if using an erasure coding scheme of 6+3, to protect erasure coded object data from a site loss, a StorageGRID Webscale deployment must include a minimum of three sites. However, a StorageGRID Webscale deployment that creates erasure coded copies may initially require more Storage Nodes than a deployment that will create replicated copies and may also require more sites. For example, a 10 MB object that is replicated once consumes 20 MB of disk space, while an object that is erasure coded with a 6+3 scheme only consumes 15 MB of disk space. Object data protected from loss through erasure coding consumes less disk space than if it is protected through replication. However, if the erasure coding scheme’s fault tolerance is breached (four in the case of a 6+3 erasure coding scheme), the object is considered lost and cannot be retrieved. This is the erasure coding scheme’s fault tolerance. Any three fragments (data or parity) can be lost and object data is still recoverable. The following is an example of a 6+3 erasure coding scheme, detailing the number of fragments that object data is broken into and the maximum number of fragments that can be lost without impacting the ability to recover an object depends on the erasure coding scheme used. For example, in a 6+3 erasure coding scheme, each part of object data is encoded into six data fragments and three parity fragments and a maximum of three fragments can be lost. An erasure code's parameters are the number of data and parity fragments generated for each part of an erasure coded object. For example, an erasure coding scheme of 6+3 creates six data fragments and three parity fragments distributed across nine Storage Nodes.Īn erasure coding scheme is defined by the erasure code's parameters and determines the maximum number of fragments that can be lost while maintaining the ability to retrieve the object. These data and parity fragments are distributed amongst the associated storage pool's Storage Nodes. The number of data and parity fragments that object data is broken into depends on the erasure coding scheme selected for the Erasure Coding profile. A data fragment is a portion of the object’s data, while a parity fragment contains the information required to reconstruct object data if data fragments are lost. If fragments are lost, object data can still be recovered through the information encoded in the remaining fragments. You then select this storage pool and its associated Erasure Coding profile when configuring an ILM rule’s Content Placement instructions to create an erasure coded copy whose fragments are distributed amongst the selected storage pool’s Storage Nodes.Įrasure coding protects object data from loss by breaking it into data and parity fragments. An Erasure Coding profile allows you to associate a storage pool with an erasure coding scheme.
0 Comments
Read More
Leave a Reply. |