batchgo

batchgo is a Golang library for batch processing data. It provides a simple way to process large amounts of data in batches.

Features

1. Asynchronous batch processing
2. concurrency-safe mechanism to add item to batch
3. Slicer interface empowers users to merge or append data

Go Docs

Documentation at pkg.go.dev

Installation

go get github.com/WheeskyJack/batchgo

Usage

The user needs to implement the Slicer interface on the batching object for this package to work. It has Append, Len Export and OnFailure methods.

Append method is called to merge the item into existing batch.

Len method is called to get the length of current batch.

Export method is called once batch is full and ready for processing.

OnFailure method is called if Export method fails.

Please refer to the example directory for a working demonstration. It showcases how Add() can be called concurrently for batching.

Contributing

Contributions are welcome! Please feel free to create an issue and submit a pull request.

License

This project is licensed under the MIT License - see the LICENSE file for details.