CSVProcess writes the contents to the file and optionally sends the written bytes upstream on outputChan.

CSVString returns an empty string for nil values to make sure that the text "null" is not written to a file.

DeleteS3Objects deletes the objects specified by the given object keys.

ExecuteSQLQuery allows you to execute arbitrary SQL statements.

GetDataFromSQLQuery is a util function that, given a properly intialized sql.DB and a valid SQL query, will handle executing the query and getting back data.JSON objects.

GetS3Object returns the object output for the given object key.

KillPipelineIfErr is an error-checking helper.

ListS3Objects returns all object keys matching the given prefix.

NewCSVWriter instantiates a new instance of CSVWriter.

SftpClient sets up and return the client.

SftpKeyAuth generates an ssh.AuthMethod given the path of a private key.

SQLInsertData abstracts building and executing a SQL INSERT statement for the given Data object.

StartTimer returns a new Timer that's already "started".

WriteS3Object writes the data to the given key, optionally compressing it first.

CSVParameters allows you to define all of your csv writing preferences in a single struct for reuse in multiple processors.

CSVWriter reimplements the standard library csv.Writer adding AlwaysEncapsulate and QuoteEscape.

SftpParameters is used for storing connection parameters for later executing sftp commands.

SftpPath is a simple struct for storing the full path of an object.

Timer is a basic mechanism for measuring execution time.