Categorygithub.com/overmindtech/sdp-go
modulepackage
0.93.1
Repository: https://github.com/overmindtech/sdp-go.git
Documentation: pkg.go.dev

# README

SDP Go Libraries

Go Reference

A set of Golang libraries for State Description Protocol

Auth

These libraries contain an EnsureValidToken HTTP middleware that can be used as follows:

router := http.NewServeMux()
router.Handle(withCORS(
    sdp.EnsureValidTokenWithPattern(
        sdpconnect.NewBookmarksServiceHandler(
            &bookmarkHandler,
            connect.WithInterceptors(otelconnect.NewInterceptor(otelconnect.WithTrustRemote(), otelconnect.WithoutTraceEvents())),
        ))))
router.Handle(withCORS(
    sdp.EnsureValidTokenWithPattern(
        sdpconnect.NewSnapshotsServiceHandler(
            &snapshotHandler,
            connect.WithInterceptors(otelconnect.NewInterceptor(otelconnect.WithTrustRemote(), otelconnect.WithoutTraceEvents())),
        ))))

serverAddress := fmt.Sprintf(":%v", "8080")
gatewayHTTPServer = &http.Server{
    Addr:    serverAddress,
    Handler: router,
}

err := gatewayHTTPServer.ListenAndServe()

Note however that using this will require the following environment variables to be present:

NameDescription
AUTH0_DOMAINThe domain to validate token against e.g. om-dogfood.eu.auth0.com
AUTH0_AUDIENCEThe audience e.g. https://api.overmind.tech
AUTH_COOKIE_NAME(Optional) The name of the cookie to extract a token from if not present in the Authorization header

Linked Item Query Extraction

This package provides some helper methods to extract linked items from unknown data structures. This is intended to be used for sections of config that are likely to have interesting data, but in a format that we don't know about. A good example would be a the env vars of a kubernetes pod.

This supports extracting the following formats:

  • IP addresses
  • HTTP/HTTPS URLs
  • DNS names
  • AWS ARNs

ExtractLinksFromAttributes

func ExtractLinksFromAttributes(attributes *ItemAttributes) []*LinkedItemQuery

This function attempts to extract linked item queries from the attributes of an item. It is designed to be used on items known to potentially contain references that can be discovered, but are in an unstructured format from which linked item queries cannot be directly constructed.

ExtractLinksViaJSON

func ExtractLinksViaJSON(i any) ([]*LinkedItemQuery, error)

This function performs the same operation as ExtractLinksFromAttributes, but takes any input format and converts it to a map[string]interface{} via JSON. It then extracts the linked item queries in a similar manner to ExtractLinksFromAttributes.

# Packages

No description provided by the author
This was written as part of an experiment That required the use of the pagerank algorithm on Overmind data.
This package contains utilities that are used to feed SDP data back and forth into an LLM.
No description provided by the author
No description provided by the author

# Functions

AddBypassAuthConfig Adds the requires keys to the context so that authentication is bypassed.
CalculatePaginationOffsetLimit Calculates the offset and limit for pagination in SQL queries, along with the current page and total pages that should be included in the response This also sets sane defaults for the page size if pagination is not provided.
ExtractAccount Extracts the account name from a context.
The same as `ExtractLinksFromAttributes`, but takes any input format and converts it to a set of ItemAttributes via the `ToAttributes` function.
This function tries to extract linked item queries from the attributes of an item.
HasAllScopes checks that the authenticated user in the request context has all the required scopes.
HasAnyScopes checks that the authenticated user in the request context has any of the required scopes.
HasScopes compatibility alias for HasAllScopes.
No description provided by the author
LogRecoverToReturn Recovers from a panic, logs and forwards it sentry and otel, then returns Does nothing when there is no panic.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
NewAuthMiddleware Creates new auth middleware.
No description provided by the author
No description provided by the author
No description provided by the author
NewQueryError converts a regular error to a QueryError of type OTHER.
No description provided by the author
NewQueryProgress returns a pointer to a QueryProgress object with the responders map initialized.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
NewSentryInterceptor pass this to connect handlers as `connect.WithInterceptors(NewSentryInterceptor())` to recover from panics in the handler and report them to sentry.
No description provided by the author
OverrideAuthContext overrides the authentication data and token stored in the context.
OverrideCustomClaims Overrides the custom claims in the context that have been set at CustomClaimsContextKey.
ToAttributes Converts a map[string]interface{} to an ItemAttributes object.
Converts to attributes using an additional set of custom transformers.
Converts a map[string]interface{} to an ItemAttributes object, sorting all slices alphabetically.This should be used when the item doesn't contain array attributes that are explicitly sorted, especially if these are sometimes returned in a different order.
ToAttributesViaJson Converts any struct to a set of attributes by marshalling to JSON and then back again.
Unmarshal Does a proto.Unmarshal and logs errors in a consistent way.

# Constants

Customise advanced limits.
An optimised balance between time taken and discovery.
Discovers all possible dependencies, might take a long time and discover items that are less likely to be relevant to a change.
Runs a shallow scan for dependencies.
The blast radius is being calculated.
Everything is complete.
Determining which apps are within the blast radius.
The blast radius has been calculated and is being saved.
No state has been specified.
The blast radius has been calculated, but the change has not yet started.
The change has been ended and the results have been processed.
The change is in progress.
The change has been ended, but the results have not yet been processed.
The change has been created, but the blast radius has not yet been calculated.
DefaultDrainDelay How long to wait after all is complete before draining all NATS connections.
DefaultResponseInterval is the default period of time within which responses are sent (5 seconds).
Everything is complete.
Snapshot is being saved.
Snapshot is being taken.
No state has been specified.
No description provided by the author
Not functioning.
Functioning normally.
Health state is transitioning, such as when something is first provisioned.
The health could not be determined.
Functioning, but degraded.
The user has accepted the invitation.
The user has been invited but has not yet accepted.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
There was an error getting the access token from Auth0.
Key is ready for use.
The API key has been revoked.
This means the key has been created but we have not yet received the callback from Auth0 which allows us to fetch the access token.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
NOSCOPE means that the item was not found because we don't have access to the requested scope.
NOTFOUND means that the item was not found.
This should be used of all other failure modes, such as timeouts, unexpected failures when querying state, permissions errors etc.
TIMEOUT means that the source times out when trying to query the item.
No description provided by the author
No description provided by the author
No description provided by the author
the query has been cancelled.
the query has finished with an error status.
The query has finished and all results have been sent over the wire This is a final state.
the query has been started.
the status has not been specified.
Work has been cancelled while in progress.
The query is complete.
All sources have returned errors.
The responder has not set a response in the expected interval.
The responder is still gathering data.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
The milestone has been completed.
The milestone failed.
The milestone is currently being worked on.
The milestone hasn't been reached yet.
The milestone was skipped.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
The source is healthy.
The source is starting or updating.
The source is sleeping due to inactivity.
The source is unhealthy.
No description provided by the author
Everything is complete.
Snapshot is being saved.
Snapshot is being taken.
No state has been specified.
No description provided by the author

# Variables

Enum value maps for AccountConfig_BlastRadiusPreset.
Enum value maps for AccountConfig_BlastRadiusPreset.
Enum value maps for CalculateBlastRadiusResponse_State.
Enum value maps for CalculateBlastRadiusResponse_State.
Enum value maps for ChangeStatus.
Enum value maps for ChangeStatus.
The default transforms that are used when converting to attributes.
Enum value maps for EndChangeResponse_State.
Enum value maps for EndChangeResponse_State.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
Enum value maps for Health.
Enum value maps for Health.
Enum value maps for Invite_InviteStatus.
Enum value maps for Invite_InviteStatus.
Enum value maps for ItemDiffStatus.
Enum value maps for ItemDiffStatus.
Enum value maps for KeyStatus.
Enum value maps for KeyStatus.
Enum value maps for OnboardingStatus.
Enum value maps for OnboardingStatus.
Enum value maps for QueryError_ErrorType.
Enum value maps for QueryError_ErrorType.
Enum value maps for QueryMethod.
Enum value maps for QueryMethod.
Enum value maps for QueryStatus_Status.
Enum value maps for QueryStatus_Status.
Enum value maps for ResponderState.
Enum value maps for ResponderState.
Enum value maps for Risk_Severity.
Enum value maps for Risk_Severity.
Enum value maps for RiskCalculationStatus_ProgressMilestone_Status.
Enum value maps for RiskCalculationStatus_ProgressMilestone_Status.
Enum value maps for RiskCalculationStatus_Status.
Enum value maps for RiskCalculationStatus_Status.
Enum value maps for SourceStatus.
Enum value maps for SourceStatus.
Enum value maps for StartChangeResponse_State.
Enum value maps for StartChangeResponse_State.

# Structs

No description provided by the author
This account config is stored in the `kv.Store` protobuf key-value store in the api-server database.
No description provided by the author
AccountNameContextKey is the key that is used to store the currently acting account name.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
a complete App with machine-supplied and user-supplied values.
machine-generated metadata of this app.
user-supplied properties of this app.
No description provided by the author
No description provided by the author
No description provided by the author
AuthBypassedContextKey is a key that is stored in the request context when auth is actively being bypassed, e.g.
AuthConfig Configuration for the auth middleware.
This message stores additional information on Edges (and edge-like constructs) to determine how configuration changes can impact the linked items.
The config that is used when calculating the blast radius for a change, this does not affect manually requested blast radii vie the "Explore" view or the API.
a complete Bookmark with user-supplied and machine-supplied values.
No description provided by the author
Descriptor for a bookmark.
The user-editable parts of a Bookmark.
After a bookmark is successfully stored, this reply with the new bookmark's details is sent.
No description provided by the author
No description provided by the author
The message signals that the Query with the corresponding UUID should be cancelled.
a complete Change with machine-supplied and user-supplied values.
No description provided by the author
No description provided by the author
machine-generated metadata of this change.
Represents the current state of a given health state, and the amount that it has changed.
user-supplied properties of this change.
No description provided by the author
No description provided by the author
No description provided by the author
A smaller summary of a change.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
create a new app.
No description provided by the author
creates a new bookmark.
No description provided by the author
create a new change.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
creates a new snapshot.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
CurrentSubjectContextKey is the key that is used to store the current subject attribute.
CustomClaims contains custom data we want from the token.
CustomClaimsContextKey is the key that is used to store the custom claims from the JWT.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
delete an app.
No description provided by the author
Delete the bookmark with the specified ID.
No description provided by the author
delete a change.
No description provided by the author
No description provided by the author
No description provided by the author
deletes a given snapshot.
No description provided by the author
No description provided by the author
No description provided by the author
Edge Represents a link between two items, it is not used in regular SDP queries as it's up to the client to infer the edges from the LinkedItems field, however request managed by the gateway will explicitly send edges to reduce the processing burden on the client side.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
This requests that the gateway "expands" an item.
A union of all request made to the gateway.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
Contains the status of the gateway request.
No description provided by the author
The gateway will always respond with this type of message, however the purpose of it is purely as a wrapper to the many different types of messages that the gateway can send.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
get the details of an existing app.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
gets a specific bookmark.
No description provided by the author
No description provided by the author
No description provided by the author
get the details of a specific change.
No description provided by the author
get the details of a specific change.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
get the initial data.
No description provided by the author
get the onboarding status.
No description provided by the author
No description provided by the author
No description provided by the author
get the details of a specific snapshot.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
This is the same as Item within the package with a couple of exceptions, no real reason why this whole thing couldn't be modelled in protobuf though if required.
ItemAttributes represents the known attributes for an item.
No description provided by the author
No description provided by the author
This is a list of items, like a List() would return.
No description provided by the author
No description provided by the author
An annotated reference to list linked items.
An annotated query to indicate potential linked items.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
list all changes for an app.
No description provided by the author
No description provided by the author
No description provided by the author
list all apps.
No description provided by the author
No description provided by the author
list all bookmarks.
list changes for a snapshot UUID.
No description provided by the author
list all changes in a specific status.
No description provided by the author
list all changes.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
lists all snapshots.
No description provided by the author
No description provided by the author
Ask the gateway to load the specified bookmark into the current state.
Ask the gateway to load the specified snapshot into the current state.
No description provided by the author
Metadata about the item.
complete Onboarding information with machine-supplied and user-supplied values.
machine-generated metadata of this onboarding info.
user-supplied properties of this onboarding info.
No description provided by the author
No description provided by the author
Query represents a query for an item or a list of items.
Defines how this query should behave when finding new items.
QueryError is sent back when an item query fails.
QueryProgress represents the status of a query.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
QueryStatus informs the client of status updates of all queries running in this session.
No description provided by the author
No description provided by the author
Reference to an item The uniqueness of an item is determined by the combination of: - Type - UniqueAttributeValue - Scope.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
Response is returned when a query is made.
No description provided by the author
ResponseSender is a struct responsible for sending responses out on behalf of agents that are working on that request.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
After a snapshot is successfully stored, this reply with the new snapshot's details is sent.
No description provided by the author
No description provided by the author
No description provided by the author
A source that is capable of discovering items.
No description provided by the author
No description provided by the author
Ask the gateway to store the current state as bookmark with the specified details.
Ask the gateway to store the current state as snapshot with the specified details.
TestConnection Used to mock a NATS connection for testing.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
This message is sent to the gateway to instruct it to "undo" an Expand.
This message is sent to the gateway to instruct it to "undo" a query.
Updates the account config for the user's account.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
update an expsting app.
No description provided by the author
updates an existing bookmark.
No description provided by the author
update an existing change.
No description provided by the author
No description provided by the author
update onboarding status.
No description provided by the author
No description provided by the author
updates the properties of an existing snapshot.
No description provided by the author
No description provided by the author
No description provided by the author
UserTokenContextKey is the key that is used to store the full JWT token of the user.

# Interfaces

Comparer is an object that can be compared for the purposes of sorting.
EncodedConnection is an interface that allows messages to be published to it.

# Type aliases

No description provided by the author
No description provided by the author
No description provided by the author
CompareError is returned when two Items cannot be compared because their UniqueAttributeValue() is not sortable.
No description provided by the author
No description provided by the author
Represents the health of something, the meaning of each state may depend on the context in which it is used but should be reasonably obvious.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
The error type.
QueryMethod represents the available query methods.
The error type.
ResponderState represents the state of the responder, note that both COMPLETE and ERROR are completion states i.e.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
A function that transforms one data type into another that is compatible with protobuf.
A map of types to transform functions.