High Throughput Pub/Sub Logger (RabbitMQ + Graylog)

The Go Lang program generates random Apache-Commons logs every 400ms into a RabbitMQ direct exchange queue and subscriber recieves the logs and sends them to Graylog for monitoring

(Note: This repo experiments with the concepts of RabbitMQ, specifically the direct exhange)

Disclaimer: Works only on MacOS

Pre-requisites:

• Requires MacOS with Homebrew installed (https://brew.sh/)
• Install Docker (https://docs.docker.com/docker-for-mac/install/)
• Using homebrew install flog (https://github.com/mingrammer/flog)

Steps to Run the project:

• Install and setup Go
  • https://golang.org/doc/install
  • Go Modules: https://github.com/golang/go/wiki/Modules
• go mod download to download the requirements
• go mod vendor to download to local reposistory
• Spin-up a RabbitMQ docker container from the ./RabbitMQ directory (docker-compose up -d)

• RabbitMQ console (http://localhost:15672/) default credentials UserName: user and password: bitnami

• Spin-up a Graylog Instance from the ./graylog directory (docker-compose up -d)

• Graylog console (http://localhost:9000/) default credentials UserName: admin and password: admin

Running the subscriber:

• Change to the subscriber directory   cd ./subscriber
• Start the subscriber:   go run subscriber.go -graylog=localhost:12201

Running the publisher:

• Change to the publisher directory   cd ./publisher
• Start the publisher:   go run publish.go

! Alternatively, use MakeFile to run the targets
+ Example: make -f MakeFile install

TODO:

• Dockerize the application
• Make it platform/OS independent