# README
jar
Package jar is a readily-embeddable feature-rich proxy-focused AWS-aware distributed-oriented resiliency-enabling URL-driven superlative-laced elastic application link. At its core, JAR is "just" a load-balancing proxy taking cues from HAProxy (resiliency, zero-drop restarts, performance) and Apache HTTPD (virtualize everything) while leveraging over 20 years of systems engineering experience to provide robust features with exceptional stability.
JAR has been in production use since 2018 and handles millions of connections a day across heterogeneous application stacks.
Consumers will want to 'cd cmd/jard; go build; #enjoy'
# Packages
No description provided by the author
No description provided by the author
No description provided by the author
Package dictionary provides the Dictionary interface for abstraction, and a simple stringmap implementation called SimpleDict, used for macro definition and replacement.
No description provided by the author
No description provided by the author
Package plugins provides support for Yaegi-style http.Handler "plugins", for use in JAR.
No description provided by the author
No description provided by the author
Package watcher is used to keep an eye on file system events, and trigger actions when those events are of interest.
No description provided by the author
# Functions
AccessLogHandler is a middleware that times how long requests takes, assembled an AccessLog, and logs accordingly.
AddMetrics ranges over the supplied map, adding each as a Metric to the supplied Check.
AddStatuses ranges over the supplied StatusRegistry, adding each as a Service to the supplied Check.
AuthoritativeDomainsHandler declines to handle requests that are not listed in "authoritativedomains" config.
Bootstrap assumes that the Conf object is all set, for now at least, builds the necessary subsystems and starts running.
BootstrapChan assumes that the Conf object is all set, for now at least, builds the necessary subsystems and starts running.
BuildPath does the heavy lifting to build a single path (which may result in multiple paths, but that's just bookkeeping).
BuildPaths unmarshalls the paths config, creates handler chains, and updates the mux.
BuildPools unmarshalls the pools config, creates them, and updates the pool list ConfigPoolsHealthcheckInterval will set the healthcheck interval for pool members.
ChanBootstrap assumes that the Conf object is all set, for now at least, builds the necessary subsystems and starts running.
ConnectionCounterAdd atomically adds 1 to the ConnectionCounter.
ConnectionCounterGet atomically returns the current value of the ConnectionCounter.
ConnectionCounterRemove atomically adds -1 to the ConnectionCounter.
CopyHeaders copies http headers from source to destination, it does not overide, but adds multiple headers.
CopyRequest provides a safe copy of a bodyless request into a new request.
CopyURL provides update safe copy by avoiding shallow copying User field.
DateFinisher is a Finisher that simply returns the current system datestamp as a string, for cache testing.
DumpFinisher is a special finisher that reflects a ton of request output.
DumpHandler is a special handler that ships a ton of request output to DebugLog.
ECBDecrypt takes a base64-encoded key and RawURLencoded-base64 ciphertext to decrypt, and returns the plaintext or an error.
ECBEncrypt takes a base64-encoded key and a []byte, and returns the base64-encdoded ciphertext or an error.
EndpointDecider is a Finisher that inspects the ``switchEndpointKey`` context to determine which materialized Pool should get the request.
FileExists returns true if the provided path exists, and is not a directory.
FlashEncoding returns a URL-encoded version of the provided string, with "+" additionally converted to "%2B".
FolderExists returns true if the provided path exists, and is a directory.
Forbidden is a Finisher that returns 403 for the requested Path.
GetErrorLog gets an error-type log.
GetLog gets a standard-type log.
GetLogOrDiscard gets a standard-type log, or discards the output.
GetRequestID is returns a requestID from a context, or the empty string.
GetSwitchName is a function to return the switch name in a request's context, if present.
HandleFinisher takes a Finisher HandlerFunc name, and returns the function for it and nil, or nil and and error.
HandleGenericWrapper is essentially a noop for when no tempate or remote errorhandler is defined.
HandleHandler takes a handler name, and an existing chain, and returns a new chain or an error.
HandleReload waits 5 seconds after being called, and then rebuilds the SwitchMaps.
HandleRemoteWrapper wraps errors (HTTP codes >= 400) in a pretty wrapper for client presentation, using a Worker to make a subrequest to an error-wrapping API.
HandleTemplateWrapper wraps errors (HTTP codes >= 400) in a pretty wrapper for client presentation, using a template.
InitConfig creates an config, initialized with defaults and environment-set values, and returns it.
LoadConfig read the config file and returns a config object or an error.
LogInit initializes all of the loggers based on Conf settings.
MinuteDelayer is a special finisher that waits for 60s before returning.
MinuteStreamer is a special finisher that writes the next number, once a secondish, for 60 iterations.
NewAccessFromStrings is the safest way to create a safe, valid Access type.
NewBasicAuth takes a source, realm, and list of users, returning an initialized *BasicAuth.
NewBodyByteLimit returns an initialized BodyByteLimit.
NewCacheCluster should be called at most once, and returns an initialized CacheCluster.
NewCompression returns a pointer to a Compression struct with the specified MIME-types baked in.
NewConsistentHashPool returns a primed ConsistentHashPool.
NewConsistentHashPoolOpts exposes some internal tunables, but still returns a ConsistentHashPool.
NewCORS returns an initialized CORS struct.
NewCORSFromConfig returns an initialized CORS struct from a list of origins and a config map.
NewECBDecrypter should never be used unless you know what you're doing.
NewECBEncrypter should never be used unless you know what you're doing.
NewForbiddenPaths takes a list of regexp-compatible strings, and returns the analogous ForbiddenPaths with compiled regexps, or an error if a regexp could not be compiled.
NewHMAC returns an initialized Verifier.
NewNoopResponseWriter returns a NoopResponseWriter that you almost definitely do not want to use.
NewPool returns a new, minimal, unmaterialized pool with the attached config.
NewPools creates a functioning Pools struct, initialized with the pools, and a healthcheck interval.
NewProcessInfo returns an intialized ProcessInfo that has an interval set to 1 minute.
NewRateLimiter returns a RateLimiter based on the specified max rps and purgeDuration.
NewRateLimiterCollector returns a RateLimiter based on the specified max rps and purgeDuration.
NewS3Pool returns an S3Pool or an error.
NewStickyPool returns a primed RoundRobin that honors pinning based on a cookie value.
NewSuiteMapFromCipherSuites takes a []*CipherSuite and creates a SuiteMap from it.
NewVerifiedBasicAuth takes a source, realm, and list of users, verifies the auth source, and returns an initialized *BasicAuth or an error.
OkFinisher is a Finisher that simply returns "Ok", for throughput testing.
PoolLister is a finisher to list the pools.
PoolMemberAdder is a finisher to add a member to an existing pool.
PoolMemberLister is a finisher to list the members of an existing pool.
PoolMemberLoser is a finisher to remove a member from an existing pool.
PrettyPrint returns the a JSONified version of the string, or %+v if that's not possible.
ReaderToString reads from a Reader into a Buffer, and then returns the string value of that.
RealAddr is a special handler to grab the most probable "real" client address.
Recoverer is a wrapping handler to make panic-capable handlers safer.
ReplaceURI standardizes the replacement of the Request.URL.Path and Request.RequestURI, which are squirrely at best.
RequestErrorResponse is the functional equivalent of ErrRequestError .WrappedResponse(..).
RequestErrorString is the functional equivalent of ErrRequestError .String().
RequestIDFinisher is a Finisher that simply returns the current requestID a random number of times, for grind testing.
ResponseHeaders is a simple piece of middleware that sets configured headers.
Restart signals the server to restart itself.
RouteIDInspectionHandler checks the Query params for a ROUTEID and shoves it into a cookie.
S3StreamProxyFinisher is a finisher that streams a POSTd file to an S3 bucket.
SetupHandler adds the RequestID and various other informatives to a request context.
Stack is a Finisher that dumps the current stack to the request.
StringIfCtx will return a non-empty string if the suppled Request has a Context.WithValue() of the specified name.
StringToCleanList splits 'in' using 'sep'arator, trims any leading/trailing space from each resulting string, and returns the list.
StringToHealthCheckStatus takes a string HealthCheckStatus and returns the HealthCheckStatus or ErrNoSuchHealthCheckStatus.
SwitchHandler adds URL switching information to the request context.
TestFinisher is a special finisher that outputs some detectables.
TrimPrefixURI standardizes the prefix trimming of the Request.URL.Path and Request.RequestURI, which are squirrely at best.
Unzip takes a source zip, and a destination folder, and unzips source into dest, returning an error if appropriate.
Update signals the updater to update itself.
URLCaptureHandler is an unchainable handler that captures the Hostname of the Pool Member servicing a request.
ValidateExtras runs through a list of referenced functions, and returns any errors they return.
WithRqID returns a context which knows its request ID.
WithSessionID returns a context which knows its session ID.
# Constants
CacheAlreadyDefinedError is returned when a Path has CacheName set, but that CacheName has already been used.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
ConfigCacheControlHeader is the value to append to a `Cache-Control` response header.
ConfigCacheExpiration is the duration in which an item should live in the cache.
ConfigCacheMaxItemSizeB is the size - in bytes - at which an item will not be cached.
ConfigCacheSizeMB is the size- in megabytes- the cache should be.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
ConfigGroupCacheAddr is a config key for my listening address, ala ``:8080``.
ConfigGroupCachePeers is a config for a list of peers, ala ``http://127.0.0.1:8080,http://127.0.0.1:8081,http://127.0.0.1:8082``.
Constants for configuration key strings.
Constants for configuration key strings.
PathOptions for HMAC signing.
PathOptions for HMAC signing.
PathOptions for HMAC signing.
PathOptions for HMAC signing.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
e.g.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings and Errors.
Constants for configuration key strings and Errors.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for configuration key strings.
Constants for HealthCheckStatuses.
ErrAborted is only used during panic recovery, if http.ErrAbortHandler was called.
ErrAuthError is used to communicate authentication errors.
ErrBootstrapDone should not be treated as a proper error, as it is returned if Bootstrap is complete (e.g.
ErrCiphertextIrregular is returned when the ciphertext is not a multiple of the block size.
ErrCiphertextTooShort is returned when the ciphertext is too damn short.
ErrConsistentHashInvalidSource is returned the source is not one of "request", "header", or "cookie".
ErrConsistentHashNextServerUnsupported is returned if NextServer is called.
ErrConsistentHashSourceNameImbalance is returned when the configured lists are not of the same lengths.
ErrFinisher404 returned by HandleFinisher if the requested finisher doesn't exist.
ErrForbiddenError is used to communicate resource access denial.
ErrInvalidS3URL is returned when the relevant URL parts from the provided S3 URL cannot be derived.
ErrNoSession is called when an AWS feature is called, but there is no initialized AWS session.
ErrNoSuchEntryError is returned by the Status Registry when no status exists for the requested thing.
ErrNoSuchHealthCheckStatus is returned when a string-based status has been used, but no corresponding HealthCheckStatus exists.
ErrNoSuchMemberError is returned if the member doesn't exist or has been removed from a Pool.
ErrNoZulipClient is returned by a worker when there is no Zulip client defined.
ErrPoolAddMemberNotSupported is returned when Pool.AddMember is called on a Pool that doesn't support the operation.
ErrPoolBuild is a panic in bootstrap() if BuildPools fails.
ErrPoolConfigConsistentAndSticky is returned when a Pool has both Sticky and ConsistentHashing set.
ErrPoolConfigMissing is returned when an operation on a Pool is requested, but no config is set.
ErrPoolDeleteMemberNotSupported is returned when Pool.DeleteMember is called on a Pool that doesn't support the operation.
ErrPoolNoMembersConfigured is returned when a non-dynamic Pool type (e.g.
ErrPoolRemoveMemberNotSupported is returned when Pool.RemoveMember is called on a Pool that doesn't support the operation.
ErrPoolsConfigdefaultmembererrorstatusEmpty is returned when the pools.defaultmembererrorstatus is empty.
ErrPoolsConfigdefaultmembererrorstatusInvalid is returned when the pools.defaultmembererrorstatus is set improperly.
ErrPoolStickyAESNoKey is returned when materializing a Pool with StickyCookieType set to 'aes' but 'keys.stickycookie' is not set.
ErrPreviousError is exclusive to the HandleSuddenEviction scheme, and announces that original error should be the one returned to the caller.
ErrS3ProxyConfigNoAWS is returned when the s3proxy is used, but AWS is not.
ErrSourceNotSupported is an error returned when the authentication source scheme is not supported [yet].
ErrSourceVerificationFailed is an error returned when an authentication source cannot be verified.
Constants for configuration key strings and Errors.
ErrUnknownError is returned when an error occurs for undefined-yet-anticipated reasons.
ErrUpdateConfigEmptyURL is returned when the updatepath is empty.
ErrUpdateConfigS3NoAWS is returned when the s3 updatepath is set, but AWS is not.
ErrValidateExtras is a panic in bootstrap() if there are any errors in ValidateExtras.
ErrVersion is a panic in bootstrap() if versionrequired is set in the config, but is less than the VERSION constant of the compiled binary reading the config.
NoCacheDefinedError is returned when a Path has CacheName set, but no cache has been globally configured.
NoPoolsError is returned if there are no pools, but a Build was requested.
Constants for HealthCheckStatuses.
PathOptionsKey is a keyid for setting/getting PathOptions to/from a Context.
Constants for miscellaneous conditions.
Constants for miscellaneous conditions.
Constants for HealthCheckStatuses.
Constants for HealthCheckStatuses.
# Variables
AccessOut is a log.Logger for access logging.
AddWork queues up some work for workers.
AWSSession is an aws.Session for use in various places.
Caches is a GroupCache used for various subsystems.
CheckAuthoritative compares domain suffixes in the "authoritativedomains" against the requested URL.Hostname() and returns true if it matches or if "authoritativedomains" is not used.
Ciphers is a map of ciphers from crypto/tls.
CommonOut is a log.Logger for Apache "common log format" logging.
Conf is the config struct.
ConfigAdditions is used to wire in additional default configurations.
ConfigValidations is used to wire in func()error to be run, validating distributed configs.
ConnectionCounter is used for tracking the current number of connections served.
CorsHandler is the global handler for CORS.
Counter is the clicker to a request counter.
CurrentHealthCheck is a cache of the current state, refreshed periodically.
DebugOut is a log.Logger for debug messages.
DefaultClient should be used instead of using http.DefaultClient, for pools/etc.
DefaultMemberWeight is the weight added to each member by default.
DefaultTrip should be used instead of the http.DefaultTransport, for pools/etc.
ErrorOut is a log.Logger for error messages.
ErrorTemplate is an HTML template for returning errors.
FileWatcher is an abstracted mechanism for calling WatchHandlerFuncs when a file is changed.
Finishers is a map of available HandlerFuncs.
FinisherSetups is a map of Finishers that need exec-time setup checks.
GOVERSION is the version of Go.
Handlers is a map of available Handlers (middlewares).
HealthCheck is a Finisher that writes the healthcheck.
Hostname is a local cache of os.Hostname.
InitFuncs are called in the early phases of Bootstrap().
IntSelf is a niladic that will trigger an interrupt of this process.
KillSelf is a niladic that will trigger a graceful shutdown of this process.
LoadBalancers are Pools.
LocalMemberWeight is the weight assigned to each member that is AZ-local.
LogPool is a Pool of AccessLogs.
MacroDictionary is a Dictionary for doing mcro.
Materializers is a map of available PoolMaterializers.
Metrics is a Registry for metrics, to be reported in the healthcheck.
NUMCPU is the number of CPUs at starttime.
OutFormat is a log.Logger format used by default.
RecyclableBufferPool is a pool of recyclable.Buffers that are safe to Get() and use (after a reset), and then Close() them when you're done, to put them back in the Pool.
RequestTimer is a function to allow Durations to be added to the Timer Metric.
ResponseModifierChain is a ProxyResponseModifierChain to handle sequences of modifications use ``ResponseModifierChain.Add()`` to add your own.
RestartSelf is a niladic that will trigger a graceful restart of this process.
Seq is a Sequence used for request ids.
SlowOut is a log.Logger for slow request information.
SlowRequests is the slow request log Duration.
SslVersions is a map of SSL/TLS versions, mapped locally.
Status is a Registry for statuses, to be reported in the healthcheck.
StopFuncs is an aggregator for functions that needs to be called during graceful shutdowns.
StrainFuncs is an aggregator for functions that can be called when JAR is under resource pressure.
SwitchMaps are maps of URLs parts and their IDs and/or endpoints.
TaskRegistry is for wrangling scheduled tasks.
TerseHealthCheck is a Finisher that writes the terse healthcheck.
ThisProcess is updated information about this process.
TimingOut is a log.Logger for timing-related debug messages.
VERSION is the internal code revision number.
Workers are a pool of workers.
ZulipClient is a global Zulip client to use for messaging, or nil if not.
# Structs
Access is a type to provide binary validation of addresses, based on the contents of "Allow/Deny" rules.
BasicAuth wraps a handler requiring HTTP basic auth.
BodyByteLimit is a Request.Body size limiter.
CacheCluster is our internal representation of GroupCache.
Cert encapsulated a Domain, the Keyfile, and a Certfile.
Compression is used to support GZIP compression of data en route to a client.
ConsistentHashPool is a PoolManager that implements a consistent hash on a key to return the proper member consistently.
CORS is an abstraction to handle CORS header nonsense.
DebugTrip is an http.RoundTripper with a pluggable core func to aid in debugging.
ErrConfigurationError is returned when a debilitating configuration error occurs.
An ErrorWrapper is a struct to abstract error wrapping.
ErrRequestError should be returned whenever an error is returned to a requestor.
ForbiddenPaths is a struct to assist in the expedient resolution of determining if a Request is destined to a forbidden path.
GenericResponse is a Finisher that returns a possibly-wrapped response.
HealthCheckError is an error returned through the HealthCheck system.
HealthCheckResult is a non-error returned through the HealthCheck system.
HealthCheckWork is Work to run a HealthCheck.
HMAC is a Handler that verifies the signature and possibly the timestamp of a request URL, and a Finisher that can sign URLs if so desired.
HTTPWork is a generic Work that can make HTTP requests.
JSONAccessLog is an AccessLog uberstruct for JSONifying log data.
Member is an attribute struct to describe a Pool Member.
NoopResponseWriter is a hack to support a Response with a status and headers, but no body.
PageCache is a cache that is specific to caching responses.
Path is an extensible struct, detailing its configuration.
PathHandler is a wrapping struct to inject the Path name, and any PathOptions into the Context.
PathReplacer is a wrapping struct to replace the Request path.
PathStripper is a wrapping struct to remove the prefix from the Request path.
Pool is a list of like-minded destinations.
PoolConfig is type exposing expected configuration for a pool, abstracted for passing around.
PoolID is a wrapping struct to inject the Pool name into the Context.
Pools is a goro-safe map of Pool objects, and if interval > 0, will also healthcheck pool members, managing them accordingly.
ProcessInfo is used to track information about ourselves.
ProxyResponseModifierChain is an encapsulating type to chain multiple ProxyResponseModifier funcs for sequential execution as a single ProxyResponseModifier.
RateLimiter is a wrapper around limiter.Limiter.
Redirect is a Finisher that returns 301 for the requested Path.
S3Pool is an http.Handler that grabs a file from S3 and streams it back to the client.
TemplateError is a static structure to pass into error-wrapping templates.
Timeout is a middleware that causes a 503 Service Unavailable message to be handed back if the timeout trips.
ZulipWork is a generic Work that can send Zulip notifications.
# Interfaces
AccessLog is an interface providing base logging, but allowing addons to extent it easily.
PoolManager is an interface to encompass oxy/roundrobin and our chpool.
# Type aliases
ConfigKey is a string type for static config key name consistency.
CorsString is a string type for static string consistency.
Error is an error type.
FinisherMap maps Finisher names to their HandlerFuncs.
FinisherSetupFunc is declared for Finishers that need exec-time setup checks.
HandlerMap maps handler names to their funcs.
HealthCheckStatus is a specific int for HealthCheckStatus consts.
PathOptions is an MSI with a case-agnostic getter.
PoolMaterializer is a function responsible for materializing a Pool.
PoolOptions is an MSI with a case-agnostic getter.
ProxyResponseModifier is a type interface compatible with oxy/forward, to allow the proxied response to be modified at proxy-time, before the Handlers will see the response.
PruneFunc is a func that may add or remove Pool members.
StatusFinisher is an abstracted type to dynamically provide Finishers of standard HTTP status codes.
SuiteMap is a map of TLS cipher suites, to their hex code.