# README
dshardorchestrator provides a sharding orchestrator for discord bots.
It's purpose is to manage nodes and assign shards to them, aswell as migrating shards between the nodes, which is one method of scaling large discord bots, spreading shards across processes and servers.
A here is a "process", which can be spread over hosts if you want.
I would not recommend using this, as 1. it currently only works against my discordgo fork and 2. its lacking a lot of tests still (although this im improving on)
It's currently used in YAGPDB.
Pitfalls
Currently its somewhat easy to break, if you try to break it that is, im working towards that but yeah, in it's current state i would just not recommend using it, unless you know what you're doing.
Essentials TODO:
- Full upgrade (simple function to migrate all nodes)
- Add in safeguards for doing things like, stopping nodes in the middle of migration.
Later:
- Extended status polling form nodes
# Packages
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
# Functions
ConnFromNetCon wraos a Conn around a net.Conn.
No description provided by the author
DecodePayload decodes a event payload according to the specific event.
EncodeMessage is the same as EncodeMessageRaw but also encodes the data passed using msgpack.
EncodeMessageRaw encodes the event to the wire format The wire format is pretty basic, first 4 bytes is a uin32 representing what type of event this is next 4 bytes is another uin32 which represents the length of the body next n bytes is the body itself, which can even be empty in some cases.
RegisterUserEvent registers a new user event to be used in shard migration for example calling this after opening a connection or otherwise concurrently will cause race conditions the reccomended way would be to call this in init()
panics if id is less than 100, as that's reserved id's for inernal use.
# Constants
EvtAllUserdataSent is sent with the total number of user data events sent.
EvtIdentified is a response to EvtIdentify orchestrator -> node: The connection was sucessfully established, now ready for whatever else.
EvtIdentify identify new node connection orchestrator <- node: Identify the new connection, orchestrator responds with a EvtIdentified if successfulll.
EvtPrepareShardmigration is sent from the orchestrator when we should prepare for a shard migration, and also used as a response orchestrator -> origin node: close the gateway connection and respond with a EvtPrepareShardmigration with session ID and sequence number orchestrator <- origin node: send when the origin node has closed the gateway connection, includes sessionID and sequence number for resuming on destination node, the event is forwarded to the destination node orchestrator -> destination node: save the session id and sequence number and prepare for a incoming shard transfe, respond with EvtPrepareShardmigration when ready orchestrator <- destination node: sent as a response when ready for the shard data transfer, followed by EvtStartShardMigration.
EvtShardMigrationDataStartID isn't an event per se, but this marks where user id's start events with higher ID than this are registered and fully handled by implementations of the node interface and will not be decoded or touched by the orchestrator.
EvtShutdown is sent to shut down a node completely, exiting orchestrator -> node: shut down the node completely.
EvtStartShardMigration is used when we should start transferring shard data, the flow goes like this: orchestrator -> oirign node: start sending all user data events, should respond with a EvtAllUserdataSent with the total number of user data events sent.
EvtStartShards assigns the following shards to the node, going through the full identify flow orchestrator -> node: assign the shards to this node, respond with the same event when processed orchestrator <- node: sent as a response when the node has been registered, does not need to have fully connected the shard yet, just registered.
EvtStopShard is sent to stop the following shard orhcestrator -> node: stop the shard, respond with the same event when done orhcestrator <- node: sent when shard has been stopped.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
# Variables
EventsToStringMap is a mapping of events to their string names.
EvtDataMap is a mapping of events to structs for their data.
No description provided by the author
# Structs
No description provided by the author
Conn represents a connection from either node to the orchestrator or the other way around it implements common logic across both sides.
No description provided by the author
No description provided by the author
Message represents a protocol message.
No description provided by the author
ShardInfo represents basic shard session info.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
UnknownEventError represents an error for unknown events, this is techincally impossible with protocol versions being enforced, but who knows if you write your own node.
# Interfaces
No description provided by the author
# Type aliases
EventType represents a dshardorchestrator protocol event The internal event IDs are hardcoded to preserve compatibility between versions.
No description provided by the author
No description provided by the author