NewBigQueryReader returns an instance of a BigQueryExtractor ready to run a static query.

NewBigQueryWriter instantiates a new instance of BigQueryWriter.

NewBigQueryWriterForNewTable instantiates a new instance of BigQueryWriter and prepares to write results to a new table.

NewCSVTransformer returns a new CSVTransformer wrapping the given io.Writer object.

NewCSVWriter returns a new CSVWriter wrapping the given io.Writer object.

NewDynamicBigQueryReader returns an instance of a BigQueryExtractor ready to run a dynamic query based on the sqlGenerator function.

NewDynamicSQLExecutor returns a new SQLExecutor operating in dynamic mode.

NewDynamicSQLReader returns a new SQLReader operating in dynamic mode.

NewDynamicSQLReaderMySQLWriter returns a new SQLReaderMySQLWriter ready for dynamic querying.

NewDynamicSQLReaderPostgreSQLWriter returns a new SQLReaderPostgreSQLWriter ready for dynamic querying.

NewFileReader returns a new FileReader that will read the entire contents of the given file path and send it at once.

NewFtpWriter instantiates new instance of an ftp writer.

NewFuncTransformer instantiates a new instance of func transformer.

NewHTTPRequest creates a new HTTPRequest and is essentially wrapping net/http's NewRequest function.

NewIoReader returns a new IoReader wrapping the given io.Reader object.

NewIoReaderWriter returns a new IoReaderWriter wrapping the given io.Reader object.

NewIoWriter returns a new IoWriter wrapping the given io.Writer object.

NewMySQLWriter returns a new MySQLWriter.

NewPassthrough instantiates a new instance of Passthrough.

NewPostgreSQLWriter returns a new PostgreSQLWriter.

NewRedshiftProcessor returns a reference to a new Redshift Processor.

NewRegexpMatcher returns a new RegexpMatcher initialized with the given pattern to match.

NewS3ObjectReader reads a single object from the given S3 bucket.

NewS3PrefixReader reads a all objects from the given S3 bucket that match a prefix.

NewS3Writer instaniates a new S3Writer.

NewSCP instantiates a new instance of SCP.

NewSftpReader instantiates a new sftp reader, a connection to the remote server is delayed until data is recv'd by the reader By default, the connection to the remote client will be closed in the Finish() func.

NewSftpReaderByClient instantiates a new sftp reader using an existing connection to the remote server.

NewSftpWriter instantiates a new sftp writer, a connection to the remote server is delayed until data is recv'd by the writer By default, the connection to the remote client will be closed in the Finish() func.

NewSftpWriterByFile allows you to manually manage the connection to the remote file object.

NewSQLExecutor returns a new SQLExecutor.

NewSQLReader returns a new SQLReader operating in static mode.

NewSQLReaderMySQLWriter returns a new SQLReaderMySQLWriter ready for static querying.

NewSQLReaderPostgreSQLWriter returns a new SQLReaderPostgreSQLWriter ready for static querying.

BigQueryConfig is used when init'ing new BigQueryReader instances.

BigQueryReader is used to query data from Google's BigQuery, and it behaves similarly to SQLReader.

BigQueryWriter is used to write data to Google's BigQuery.

CSVTransformer converts etldata.Payload objects into a CSV string object and sends it on to the next stage.

CSVWriter is handles converting etldata.JSON objects into CSV format, and writing them to the given io.Writer.

FileReader opens and reads the contents of the given filename.

FtpWriter type represents an ftp writter processor.

FuncTransformer executes the given function on each data payload, sending the resuling data to the next stage.

HTTPRequest executes an HTTP request and passes along the response body.

IoReader wraps an io.Reader and reads it.

IoReaderWriter performs both the job of a IoReader and IoWriter.

IoWriter wraps any io.Writer object.

MySQLWriter handles INSERTing etldata.JSON into a specified SQL table.

Passthrough simply passes the data on to the next stage.

PostgreSQLWriter handles INSERTing etldata.JSON into a specified SQL table.

RedshiftWriter gets data into a Redshift table by first uploading data batches to S3.

RegexpMatcher checks if incoming data matches the given Regexp, and sends it on to the next stage only if it matches.

S3Reader handles retrieving objects from S3.

S3Writer sends data upstream to S3.

SCP executes the scp command, sending the given file to the given destination.

SftpReader reads a single object at a given path, or walks through the directory specified by the path (SftpReader.Walk must be set to true).

SftpWriter is an inline writer to remote sftp server.

SQLExecutor runs the given SQL and swallows any returned data.

SQLReader runs the given SQL and passes the resulting data to the next stage of processing.

SQLReaderMySQLWriter performs both the job of a SQLReader and MySQLWriter.

SQLReaderPostgreSQLWriter performs both the job of a SQLReader and PostgreSQLWriter.

SQLWriterData is a custom data structure you can send into a MySQLWriter stage or a PostreSQLWriter stage if you need to specify TableName on a per-data payload basis.