pulse

an eventbus made on portable MQ.

Small Roadmap

For Version 3.0. Let's refer https://cloud.google.com/pubsub/docs/choosing-pubsub-or-cloud-tasks

Refactor the interface, make it clear and more easier to realize.
protobuf support for event
Metrics support for Prometheus by zap logger
Integration for cadence workflow

There are three things stable in computer industry. Storage, Network and OperateSystem. Sidecar, Or Microservice Orchestration is about the protocl layer. Sidecar would transfer and transport, in order to solve multi complex network problem in real world. Let's explore new world but not promise communication model, eventbus base on cloud message queue. In QoS 1, it like UDP protocol. With ack, like TCP. With webhook, like service invocation. So eazily scalable on engineering language--Go, and adapted with Hybrid Cloud is my goal.

The most important component is scheduler, then the driver.

smallermap

Local EventBus (More EventBus)
WebHook Support
ACK support
[ ]

Usage

Check the example/pulse and find an example for supported MQ. Note: You need run your MQ first and get its address.

Use MQ as broker

example for nats.

meta := mq.NewMetadata()
meta.Properties[nats.URL] = nats.DefaultURL
meta.Properties["DriverName"] = "nats"

Publisher

Publisher is asynchronously and could get result about the success or failure to send the event.

t, err := topic.NewTopic("hello", *meta, topic.WithRequiredACK(), topic.WithOrdered())
if err != nil {
    log.Println(err)
    return
}

res := t.Publish(context.Background(), message.NewMessage([]byte("hello")))
go func() {
    if err := res.Get(context.Background()); err != nil {
        log.Println("----------------------", err)
    }
}()

Receive is a synchronous function and blocks until have an err set by like ctx.Done() or other error.

s, err := subscription.NewSubscription("hello", *meta, subscription.WithCount(), subscription.WithAutoACK())
if err != nil {
    log.Println(err)
    return
}

err = s.Receive(context.Background(), func(ctx context.Context, m *message.CloudEventsEnvelope) {
    log.Println("receive the protocol:", m.Id)
})

feature

Idempotence: Simply record in the map of each Subscription to avoid repeated processing by a single consumer. Nats can provide queueSubscribe
Orderliness: Messages use OrderingKey to ensure orderly delivery. If an error occurs, the sending of a certain OrderingKey will be suspended, and the empty key will not be suspended
Concurrent processing: Both topic and Subscription use concurrent processing. Pay attention to whether the middleware has critical resources
Reliability: ack is implemented independently to ensure that the message is delivered at least once. In future: support QoS 0,1,2 three level.
Asynchronous: Message send asynchronously and could be buffered and delay send.
Batch Handle: Scheduler could buffer message and batch handle them if underlying MQ supports.

To-DO

You could find real-time taskboard on https://github.com/silverswords/pulse/projects

Refactor the interface, make it clear and more easier to realize.
pub/sub system but not a sdk
protobuf support for event
Metrics support for Prometheus by zap logger?
Integration for cadence workflow

Architecture

Concepts

Driver

Driver is the realization of various protocol like Nats, http etc.

Message Format

Message is the object transport from the pulse endpoint. It's format as CloudEvents.

            {
              "specversion": "1.x-wip",
              "type": "coolevent",
              "id": "xxxx-xxxx-xxxx",
              "source": "bigco.com",
              "data": { ... }
            }

Reference

Hall of fame

Repos

- nuid: now use nats's package nuid to generated uuid

- CloudEvent: a CNCF project

- Cloud State: sidecar project to move state out of application

- GoogleCloud Pub sub : use to get a new driver

- Dapr: sidecar Synthesizer

- Saga: pulse usage.

- Kong: siprit on extension.

- Kafka: log

- axon: event source DDD CQRS

- webhook: to establish a webhook to receive the response asynchronously

# README