CompressObjects takes an object storage prefix and a destination.

Download an object from a remote cloud by breaking it up and downloading it in multiple chunks concurrently.

NewChunkReader creates a new chunk reader which will read chunks of the given size from the provided reader.

NewMPDownloader creates a new multipart downloader using the given objects.

NewMPUploader creates a new multipart uploader using the given options, this will create a new multipart upload if one hasn't already been provided.

NewSyncer creates a new syncer using the given options.

ParseCloudOrFileURL parses a URL which is either a file path or a cloud path.

PrefixExists returns a boolean indicating whether any objects exist in the remote provider that have the given prefix.

Sync copies a directory to/from cloud storage from/to a filepath.

Upload an object to a remote cloud breaking it down into a multipart upload if the body is over a given size.

MaxUploadParts is the hard limit on the number of parts that can be uploaded by a 'MPUploader'.

MinPartSize is the minimum size allowed for 'PartSize', and is a hard limit enforced by AWS.

MPUThreshold is the threshold at which point we break the upload up into multiple requests which are executed concurrently.

ErrMPUploaderAlreadyStopped is returned if the upload is stopped multiple times.

ErrMPUploaderExceededMaxPartCount is returned if the user attempts to upload more than 'MaxUploadParts' parts.

ChunkReader allows data from an 'io.Reader' in chunks of a given size.

CloudOrFileURL represents a cloud storage url (eg s3://bucket/path/to/file.txt) or a local path.

CompressObjectsOptions specifies options which configure what and how objects are compressed and uploaded.

DownloadOptions encapsulates the options available when using the 'Download' function to download data from a remote cloud.

ErrInvalidCloudPath returns if the user has incorrectly used the cloud style scheme prefixed argument; the error message indicates/display the correct usage to the user.

MPDownloader is a multipart downloader which downloads an object from a remote cloud by performing multiple requests concurrently using a worker pool.

MPDownloaderOptions encapsulates the options available when creating a 'MPDownloader'.

MPUploader is a multipart uploader which adds parts to a remote multipart upload whilst concurrently uploading data using a worker pool.

MPUploaderOptions encapsulates the options available when creating a 'MPUploader'.

Options contains common options for upload/download of objects.

PrefixExistsOptions encapsulates the options available when running 'PrefixExists'.

Syncer exposes the ability to sync files and directories to/from a remote cloud provider.

SyncOptions encapsulates all the options available when syncing a directory/object to/from a remote cloud.

UploadOptions encapsulates the options available when using the 'Upload' function to upload data to a remote cloud.

OnPartCompleteFunc is a readability wrapper around a callback function which may be run after each part has been uploaded.

PartCompleteFunc is called once a part of the zip file has been uploaded.

ProgressReportFunc is called every time a file has been fully downloaded during CompressUpload.