Categorycode.cloudfoundry.org/go-pubsub
modulepackage
0.0.0-20250319120402-1dce9d2f0d04
Repository: https://github.com/cloudfoundry/go-pubsub.git
Documentation: pkg.go.dev

# README

pubsub

GoDoc

PubSub publishes data to subscriptions. However, it can do so much more than just push some data to a subscription. Each subscription is placed in a tree. When data is published, it traverses the tree and finds each interested subscription. This allows for sophisticated filters and routing.

If you have any questions, or want to get attention for a PR or issue please reach out on the #logging-and-metrics channel in the cloudfoundry slack

Installation

go get code.cloudfoundry.org/go-pubsub

Subscription Trees

A subscription tree is a collection of subscriptions that are organized based on what data they want published to them. When subscribing, a path is provided to give directions to PubSub about where to store the subscription and what data should be published to it.

So for example, say there are three subscriptions with the following paths:

NamePath
sub-1["a", "b", "c"]
sub-2["a", "b", "d"]
sub-3["a", "b"]

After both subscriptions have been registered PubSub will have the following subscription tree:

              a
              |
              b   <-(sub-3)
             / \
(sub-1)->   c   d   <-(sub-2)

To better draw out each's subscriptions view of the tree:

Sub-1
              a
              |
              b
             /
(sub-1)->   c
Sub-2
              a
              |
              b
               \
                d   <-(sub-2)
Sub-3
              a
              |
              b   <-(sub-3)

So to give a few exapmles of how data could be published:

Single path
              a
              |
              b   <-(sub-3)
               \
                d   <-(sub-2)

In this example both sub-2 and sub-3 would have the data written to it.

Multi-Path
              a
              |
              b   <-(sub-3)
             / \
(sub-1)->   c   d   <-(sub-2)

In this example all sub-1, sub-2 and sub-3 would have the data written to it.

Shorter Path
              a
              |
              b   <-(sub-3)

In this example only sub-3 would have the data written to it.

Other Path
              x
              |
              y

In this example, no subscriptions would have data written to them.

Simple Example:

ps := pubsub.New()
subscription := func(name string) pubsub.SubscriptionFunc {
	return func(data interface{}) {
		fmt.Printf("%s -> %v\n", name, data)
	}
}

ps.Subscribe(subscription("sub-1"), []string{"a", "b", "c"})
ps.Subscribe(subscription("sub-2"), []string{"a", "b", "d"})
ps.Subscribe(subscription("sub-3"), []string{"a", "b", "e"})

ps.Publish("data-1", pubsub.LinearTreeTraverser([]string{"a", "b"}))
ps.Publish("data-2", pubsub.LinearTreeTraverser([]string{"a", "b", "c"}))
ps.Publish("data-3", pubsub.LinearTreeTraverser([]string{"a", "b", "d"}))
ps.Publish("data-3", pubsub.LinearTreeTraverser([]string{"x", "y"}))

// Output:
// sub-1 -> data-2
// sub-2 -> data-3

In this example the LinearTreeTraverser is used to traverse the tree of subscriptions. When an interested subscription is found (in this case sub-1 and sub-2 for data-2 and data-3 respectively), the subscription is handed the data.

More complex examples can be found in the examples directory.

TreeTraversers

A TreeTraverser is used to traverse the subscription tree and find what subscriptions should have the data published to them. There are a few implementations provided, however it is likely a user will need to implement their own to suit their data.

When creating a TreeTraverser it is important to note how the data is structured. A TreeTraverser must be deterministic and ideally stateless. The order the data is parsed and returned (via Traverse()) must align with the given path of Subscribe().

This means if the TreeTraverser intends to look at field A, then B, and then finally C, then the subscription path must be A, B and then C (and not B, A, C or something).

Subscriptions

A Subscription is used when publishing data. The given path is used to determine it's placement in the subscription tree.

Code Generation

The tree traversers and subscriptions are quite complicated. Laying out a tree structure is not something humans are going to find natural. Therefore a generator is provided for structs.

The struct is inspected (at go generate time) and creates the tree layout code. There is a provided example.

# Packages

No description provided by the author
No description provided by the author

# Functions

CombinePaths takes several paths and flattens it into a single path.
FlatPaths implements Paths for a slice of paths.
LinearTreeTraverser implements TreeTraverser on behalf of a slice of paths.
New constructs a new PubSub.
PathsWithTraverser implement Paths and allow a TreeTraverser to have multiple paths with multiple traversers.
PathsWithTraverser implements Paths for both a slice of paths and a single TreeTraverser.
WithDeterministicHashing configures a PubSub that will use the given function to hash each published data point.
WithDeterministicRouting configures a subscription to have a deterministic routing name.
WithNoMutex configures a PubSub that does not have any internal mutexes.
WithPath configures a subscription to reside at a path.
WithRand configures a PubSub that will use the given function to make sharding decisions.
WithShardID configures a subscription to have a shardID.

# Structs

PathAndTraverser is a path and traverser pair.
PubSub uses the given SubscriptionEnroller to create the subscription tree.

# Interfaces

PubSubOption is used to configure a PubSub.
SubscribeOption is used to configure a subscription while subscribing.

# Type aliases

Paths is returned by a TreeTraverser.
Subscription is a subscription that will have corresponding data written to it.
TreeTraverser publishes data to the correct subscriptions.
Unsubscriber is returned by Subscribe.