# README

go-ipfs-chunker

go-ipfs-chunker implements data Splitters for go-ipfs.

❗ This repo is no longer maintained.

👉 We highly recommend switching to the maintained version at https://github.com/ipfs/boxo/tree/main/chunker. 🏎️ Good news! There is tooling and documentation to expedite a switch in your repo.

⚠️ If you continue using this repo, please note that security fixes will not be provided (unless someone steps in to maintain it).

📚 Learn more, including how to take the maintainership mantle or ask questions, here.

Summary

go-ipfs-chunker provides the Splitter interface. IPFS splitters read data from a reader an create "chunks". These chunks are used to build the ipfs DAGs (Merkle Tree) and are the base unit to obtain the sums that ipfs uses to address content.

The package provides a SizeSplitter which creates chunks of equal size and it is used by default in most cases, and a rabin fingerprint chunker. This chunker will attempt to split data in a way that the resulting blocks are the same when the data has repetitive patterns, thus optimizing the resulting DAGs.

Install
Usage
License

Install

go-ipfs-chunker works like a regular Go module:

> go get github.com/ipfs/go-ipfs-chunker

Usage

import "github.com/ipfs/go-ipfs-chunker"

Check the GoDoc documentation

License

MIT © Protocol Labs, Inc.

# Packages

gen

This file generates bytehash LUT.

# Functions

Chan

Chan returns a channel that receives each of the chunks produced by a splitter, along with another one for errors.

DefaultSplitter

DefaultSplitter returns a SizeSplitter with the DefaultBlockSize.

FromString

FromString returns a Splitter depending on the given string: it supports "default" (""), "size-{size}", "rabin", "rabin-{blocksize}", "rabin-{min}-{avg}-{max}" and "buzhash".

NewBuzhash

Deprecated: use github.com/ipfs/boxo/chunker.NewBuzhash.

NewRabin

NewRabin creates a new Rabin splitter with the given average block size.

NewRabinMinMax

NewRabinMinMax returns a new Rabin splitter which uses the given min, average and max block sizes.

NewSizeSplitter

NewSizeSplitter returns a new size-based Splitter with the given block size.

SizeSplitterGen

SizeSplitterGen returns a SplitterGen function which will create a splitter with the given size when called.

# Constants

ChunkSizeLimit

No leaf block should contain more than 1MiB of payload data ( wrapping overhead aside ) This effectively mandates the maximum chunk size See discussion at https://github.com/ipfs/go-ipfs-chunker/pull/21#discussion_r369124879 for background Deprecated: use github.com/ipfs/boxo/chunker.ChunkSizeLimit.

DefaultBlockSize

DefaultBlockSize is the chunk size that splitters produce (or aim to).