DecodeBlob takes a blob read from a pack file (as per the specs from a BlobLocation) and returns the chunk stored in that blob.

Given a reader for a pack file, decodes it into blobs and then calls the given callback function for each blob's chunk.

HashBytes computes the SHAKE256 hash of the given byte slice.

NewCompressed returns a new storage.Backend that applies gzip compression to the contents of chunks stored in the provided underlying backend.

NewDisk returns a new storage.Backend that stores data to the given dir.

NewEncrypted returns a storage.Backend that applies AES encryption to the chunk data stored in the underlying storage.Backend.

NewHashesReader returns an io.ReadCloser that reads multiple hashes in parallel from the given storage backend.

NewMemory returns a storage.Backend that stores all data--blobs, hashes, metadata, etc., in RAM.

PackBLob takes (hash, chunk) pairs and the current size of the pack file and converts them to the representation to be stored in index and pack files, returning the bytes to append to the index and back files to store the chunk.

Split the bytes of the given io.Reader using a rolling checksum into chunks of size (on average) 1<<splitBits.

HashSize is the number of bytes in the hash values returned to represent chunks of data.

External representation of the location of a blob in a pack file that's returned to callers.

ChunkIndex maintains an index from hashes to the locations of their blobs in pack files.

The lowest bits seem to be most useful; splitting based on, say, 4 bits in the middle is fiddly, especially when it spans the 16th bit.

PackFileBackend implements the storage.Backend interface, but depends on an implementation of the FileStorage interface to handle the mechanics of storing and retrieving files.

Backend describes a general interface for low-level data storage; users can provide chunks of data that a storage backend will store (on disk, in the cloud, etc.), and are returned a Hash that identifies each such chunk.

FileStorage is a simple abstraction for a storage system.

RobustWriteCloser is like a io.WriteCloser, except it treats any errors as fatal errors and thus doesn't have error return values.

Hash encodes a fixed-size secure hash of a collection of bytes.