Categorygithub.com/HouMYt/go-libp2p-pubsub
modulepackage
0.5.1
Repository: https://github.com/houmyt/go-libp2p-pubsub.git
Documentation: pkg.go.dev

# README

go-libp2p-pubsub


This repo contains the canonical pubsub implementation for libp2p. We currently provide three message router options:

  • Floodsub, which is the baseline flooding protocol.
  • Randomsub, which is a simple probabilistic router that propagates to random subsets of peers.
  • Gossipsub, which is a more advanced router with mesh formation and gossip propagation. See spec and implementation for more details.

PSA: The Hardening Extensions for Gossipsub (Gossipsub V1.1) can be found under development at https://github.com/libp2p/go-libp2p-pubsub/pull/263

Repo Lead Maintainer

@vyzo

This repo follows the Repo Lead Maintainer Protocol

Table of Contents

Install

go get github.com/libp2p/go-libp2p-pubsub

Usage

To be used for messaging in p2p instrastructure (as part of libp2p) such as IPFS, Ethereum, other blockchains, etc.

Example

https://github.com/libp2p/go-libp2p-examples/tree/master/pubsub

Documentation

See the libp2p specs for high level documentation and godoc for API documentation.

In this repo, you will find

.
├── LICENSE
├── README.md
# Regular Golang repo set up
├── codecov.yml
├── pb
├── go.mod
├── go.sum
├── doc.go
# PubSub base
├── pubsub.go
├── blacklist.go
├── notify.go
├── comm.go
├── discovery.go
├── sign.go
├── subscription.go
├── topic.go
├── trace.go
├── tracer.go
├── validation.go
# Floodsub router
├── floodsub.go
# Randomsub router
├── randomsub.go
# Gossipsub router
├── gossipsub.go
├── score.go
├── score_params.go
└── mcache.go

Tracing

The pubsub system supports tracing, which collects all events pertaining to the internals of the system. This allows you to recreate the complete message flow and state of the system for analysis purposes.

To enable tracing, instantiate the pubsub system using the WithEventTracer option; the option accepts a tracer with three available implementations in-package (trace to json, pb, or a remote peer). If you want to trace using a remote peer, you can do so using the traced daemon from go-libp2p-pubsub-tracer. The package also includes a utility program, tracestat, for analyzing the traces collected by the daemon.

For instance, to capture the trace as a json file, you can use the following option:

pubsub.NewGossipSub(..., pubsub.NewEventTracer(pubsub.NewJSONTracer("/path/to/trace.json")))

To capture the trace as a protobuf, you can use the following option:

pubsub.NewGossipSub(..., pubsub.NewEventTracer(pubsub.NewPBTracer("/path/to/trace.pb")))

Finally, to use the remote tracer, you can use the following incantations:

// assuming that your tracer runs in x.x.x.x and has a peer ID of QmTracer
pi, err := peer.AddrInfoFromP2pAddr(ma.StringCast("/ip4/x.x.x.x/tcp/4001/p2p/QmTracer"))
if err != nil {
  panic(err)
}

tracer, err := pubsub.NewRemoteTracer(ctx, host, pi)
if err != nil {
  panic(err)
}

ps, err := pubsub.NewGossipSub(..., pubsub.WithEventTracer(tracer))

Contribute

Contributions welcome. Please check out the issues.

Check out our contributing document for more information on how we work, and about contributing in general. Please be aware that all interactions related to multiformats are subject to the IPFS Code of Conduct.

Small note: If editing the README, please conform to the standard-readme specification.

License

The go-libp2p-pubsub project is dual-licensed under Apache 2.0 and MIT terms:

# Packages

No description provided by the author
No description provided by the author

# Functions

DefaultMsgIdFn returns a unique ID of the passed Message.
DefaultPeerGaterParams creates a new PeerGaterParams struct using default values.
FilterSubscriptions filters (and deduplicates) a list of subscriptions.
MinTopicSize returns a function that checks if a router is ready for publishing based on the topic size.
NewAllowlistSubscriptionFilter creates a subscription filter that only allows explicitly specified topics for local subscriptions and incoming peer subscriptions.
NewFloodSub returns a new PubSub object using the FloodSubRouter.
NewFloodsubWithProtocols returns a new floodsub-enabled PubSub objecting using the protocols specified in ps.
NewGossipSub returns a new PubSub object using GossipSubRouter as the router.
NewJsonTracer creates a new JSONTracer writing traces to file.
NewMapBlacklist creates a new MapBlacklist.
NewMessageCache creates a sliding window cache that remembers messages for as long as `history` slots.
No description provided by the author
NewPeerGaterParams creates a new PeerGaterParams struct, using the specified threshold and decay parameters and default values for all other parameters.
NewPubSub returns a new PubSub management object.
NewRandomSub returns a new PubSub object using RandomSubRouter as the router.
NewRegexpSubscriptionFilter creates a subscription filter that only allows topics that match a regular expression for local subscriptions and incoming peer subscriptions.
NewRemoteTracer constructs a RemoteTracer, tracing to the peer identified by pi.
NewTimeCachedBlacklist creates a new TimeCachedBlacklist with the given expiry duration.
OpenJSONTracer creates a new JSONTracer, with explicit control of OpenFile flags and permissions.
OpenPBTracer creates a new PBTracer, with explicit control of OpenFile flags and permissions.
ScoreParameterDecay computes the decay factor for a parameter, assuming the DecayInterval is 1s and that the value decays to zero if it drops below 0.01.
ScoreParameterDecay computes the decay factor for a parameter using base as the DecayInterval.
WithBlacklist provides an implementation of the blacklist; the default is a MapBlacklist.
WithDirectConnectTicks is a gossipsub router option that sets the number of heartbeat ticks between attempting to reconnect direct peers that are not currently connected.
WithDirectPeers is a gossipsub router option that specifies peers with direct peering agreements.
WithDiscoverConnector adds a custom connector that deals with how the discovery subsystem connects to peers.
WithDiscovery provides a discovery mechanism used to bootstrap and provide peers into PubSub.
WithDiscoveryOpts passes libp2p Discovery options into the PubSub discovery subsystem.
WithEventTracer provides a tracer for the pubsub system.
WithFloodPublish is a gossipsub router option that enables flood publishing.
WithMaxMessageSize sets the global maximum message size for pubsub wire messages.
WithMessageAuthor sets the author for outbound messages to the given peer ID (defaults to the host's ID).
WithMessageIdFn is an option to customize the way a message ID is computed for a pubsub message.
WithMessageSignaturePolicy sets the mode of operation for producing and verifying message signatures.
WithMessageSigning enables or disables message signing (enabled by default).
WithNoAuthor omits the author and seq-number data of messages, and disables the use of signatures.
WithPeerExchange is a gossipsub router option that enables Peer eXchange on PRUNE.
WithPeerGater is a gossipsub router option that enables reactive validation queue management.
WithPeerOutboundQueueSize is an option to set the buffer size for outbound messages to a peer We start dropping messages to a peer if the outbound queue if full.
WithPeerScore is a gossipsub router option that enables peer scoring.
WithPeerScoreInspect is a gossipsub router option that enables peer score debugging.
WithReadiness returns a publishing option for only publishing when the router is ready.
WithStrictSignatureVerification is an option to enable or disable strict message signing.
WithSubscriptionFilter is a pubsub option that specifies a filter for subscriptions in topics of interest.
WithValidateQueueSize sets the buffer of validate queue.
WithValidateThrottle sets the upper bound on the number of active validation goroutines across all topics.
WithValidateWorkers sets the number of synchronous validation worker goroutines.
WithValidatorConcurrency is an option that sets the topic validator throttle.
WithValidatorInline is an option that sets the validation disposition to synchronous: it will be executed inline in validation front-end, without spawning a new goroutine.
WithValidatorTimeout is an option that sets a timeout for an (asynchronous) topic validator.
WrapLimitSubscriptionFilter wraps a subscription filter with a hard limit in the number of subscriptions allowed in an RPC message.

# Constants

AcceptNone signals to drop the incoming RPC.
AcceptControl signals to accept the incoming RPC only for control message processing by the router.
AcceptAll signals to accept the incoming RPC for full processing.
No description provided by the author
No description provided by the author
DefaultMaximumMessageSize is 1mb.
No description provided by the author
No description provided by the author
GossipSubID_v10 is the protocol ID for version 1.0.0 of the GossipSub protocol.
GossipSubID_v11 is the protocol ID for version 1.1.0 of the GossipSub protocol.
LaxNoSign does not produce signatures and validates incoming signatures iff one is present Deprecated: it is recommend to either strictly enable, or strictly disable, signatures.
LaxSign produces signatures and validates incoming signatures iff one is present Deprecated: it is recommend to either strictly enable, or strictly disable, signatures.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
StrictNoSign does not produce signatures and drops and penalises incoming messages that carry one.
StrictSign produces signatures and expects and verifies incoming signatures.
ValidationAccept is a validation decision that indicates a valid message that should be accepted and delivered to the application and forwarded to the network.
ValidationIgnore is a validation decision that indicates a message that should be ignored: it will be neither delivered to the application nor forwarded to the network.
ValidationReject is a validation decision that indicates an invalid message that should not be delivered to the application or forwarded to the application.

# Variables

No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
DiscoveryPollInitialDelay is how long the discovery system waits after it first starts before polling.
DiscoveryPollInterval is approximately how long the discovery system waits in between checks for whether the more peers are needed for any topic.
ErrSubscriptionCancelled may be returned when a subscription Next() is called after the subscription has been cancelled.
ErrTooManySubscriptions may be returned by a SubscriptionFilter to signal that there are too many subscriptions to process.
ErrTopicClosed is returned if a Topic is utilized after it has been closed.
GossipSubConnectionTimeout controls the timeout for connection attempts.
GossipSubConnectors controls the number of active connection attempts for peers obtained through PX.
GossipSubConnTagBumpMessageDelivery is the amount to add to the connection manager tag that tracks message deliveries.
GossipSubConnTagDecayAmount is subtracted from decaying tag values at each decay interval.
GossipSubConnTagDecayInterval is the decay interval for decaying connection manager tags.
GossipSubConnTagMessageDeliveryCap is the maximum value for the connection manager tags that track message deliveries.
GossipSubD sets the optimal degree for a GossipSub topic mesh.
GossipSubDhi sets the upper bound on the number of peers we keep in a GossipSub topic mesh.
GossipSubDirectConnectInitialDelay is the initial delay before opening connections to direct peers.
GossipSubDirectConnectTicks is the number of heartbeat ticks for attempting to reconnect direct peers that are not currently connected.
GossipSubDlazy affects how many peers we will emit gossip to at each heartbeat.
GossipSubDlo sets the lower bound on the number of peers we keep in a GossipSub topic mesh.
GossipSubDout sets the quota for the number of outbound connections to maintain in a topic mesh.
GossipSubDscore affects how peers are selected when pruning a mesh due to over subscription.
GossipSubFanoutTTL controls how long we keep track of the fanout state.
GossipSubGossipFactor affects how many peers we will emit gossip to at each heartbeat.
GossipSubGossipRetransmission controls how many times we will allow a peer to request the same message id through IWANT gossip before we start ignoring them.
If a GRAFT comes before GossipSubGraftFloodThreshold has elapsed since the last PRUNE, then there is an extra score penalty applied to the peer through P7.
GossipSubHeartbeatInitialDelay is the short delay before the heartbeat timer begins after the router is initialized.
GossipSubHeartbeatInterval controls the time between heartbeats.
GossipSubHistoryGossip controls how many cached message ids we will advertise in IHAVE gossip messages.
GossipSubHistoryLength controls the size of the message cache used for gossip.
Time to wait for a message requested through IWANT following an IHAVE advertisement.
GossipSubMaxIHaveLength is the maximum number of messages to include in an IHAVE message.
GossipSubMaxIHaveMessages is the maximum number of IHAVE messages to accept from a peer within a heartbeat.
GossipSubMaxPendingConnections sets the maximum number of pending connections for peers attempted through px.
GossipSubOpportunisticGraftPeers is the number of peers to opportunistically graft.
GossipSubOpportunisticGraftTicks is the number of heartbeat ticks for attempting to improve the mesh with opportunistic grafting.
GossipSubPruneBackoff controls the backoff time for pruned peers.
GossipSubPrunePeers controls the number of peers to include in prune Peer eXchange.
No description provided by the author
No description provided by the author
No description provided by the author
64K ought to be enough for everyone; famous last words.

# Structs

No description provided by the author
No description provided by the author
GossipSubRouter is a router that implements the gossipsub protocol.
JSONTracer is a tracer that writes events to a file, encoded in ndjson.
No description provided by the author
No description provided by the author
PBTracer is a tracer that writes events to a file, as delimited protobufs.
No description provided by the author
PeerGaterParams groups together parameters that control the operation of the peer gater.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
PubSub is the implementation of the pubsub system.
RandomSubRouter is a router that implements a random propagation strategy.
RemoteTracer is a tracer that sends trace events to a remote peer.
No description provided by the author
Subscription handles the details of a particular Topic subscription.
TimeCachedBlacklist is a blacklist implementation using a time cache.
Topic is the handle for a pubsub topic.
TopicEventHandler is used to manage topic specific events.
No description provided by the author
No description provided by the author
No description provided by the author

# Interfaces

Blacklist is an interface for peer blacklisting.
Generic event tracer interface.
PubSubRouter is the message router component of PubSub.
SubscriptionFilter is a function that tells us whether we are interested in allowing and tracking subscriptions for a given topic.

# Type aliases

No description provided by the author
BackoffConnectorFactory creates a BackoffConnector that is attached to a given host.
No description provided by the author
No description provided by the author
No description provided by the author
MapBlacklist is a blacklist implementation using a perfect map.
MessageSignaturePolicy describes if signatures are produced, expected, and/or verified.
MsgIdFunction returns a unique ID for the passed Message, and PubSub can be customized to use any implementation of this function by configuring it with the Option from WithMessageIdFn.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
RouterReady is a function that decides if a router is ready to publish.
No description provided by the author
No description provided by the author
No description provided by the author
ValidationResult represents the decision of an extended validator.
Validator is a function that validates a message with a binary decision: accept or reject.
ValidatorEx is an extended validation function that validates a message with an enumerated decision.
ValidatorOpt is an option for RegisterTopicValidator.