package
0.0.0-20240325134752-7b41e72b8503
Repository: https://github.com/snowplow/dataflow-runner.git
Documentation: pkg.go.dev
# Functions
Diff outputs the difference between two string slices where a is the reference (a - b).
GetCredentialsProvider attempts to fetch credentials from either: 1.
GetEbsConfiguration turns a EbsConfigurationRecord into an emr.EbsConfiguration.
GetLock builds a file-based or consul-based lock depending on the consul varialbe.
InitConfigResolver creates a new ConfigResolver instance.
InitConsulLock builds a ConsulLock (a KV pair in Consul) with the name argument as key.
InitEmrCluster creates a new EmrCluster instance.
InitFileLock builds a FileLock at the path speicifed by name.
InitJobFlowSteps creates a new JobFlowSteps instance.
InitLogsDownloader creates a new LogsDownloader instance.
InterfaceToJSONString writes an interface as a JSON.
ReadGzFile reads a gzipped file.
ReadGzFiles lists the files in dir and return their un-gzipped content.
StringInSlice checks whether or not a string is in an array.
# Structs
ConfigResolver is used for validating and loading all configs.
ConsulLock is for Consul-based locks.
EmrCluster is used for starting and terminating clusters.
FileLock is for file-based locks.
JobFlowSteps is used for adding steps to an existing cluster.
LogsDownloader is used to download failed steps' logs.
S3Downloader models an entity capable of downloading files from S3 at the specified bucket to the specified dir.
SelfDescribingRecord is a simple struct for loading a self-describing record.
# Interfaces
Lock interface abstracting over file-based or consul-based locks.
# Type aliases
No description provided by the author