# README

Event Horizon

Summary

Enables log-based architectures with realtime streaming, unlimited storage, compression and encryption.

Key facts
Use cases	Eventsourcing, high availability software/datacenter failover, microservices, big data.
Is this a good fit for me?	For log-based architectures, yes. Read more
Demo application	eventhorizon-exampleapp-go
Production ready?	No, not yet. But not too far away.
Homepage & documentation	Both project homepage and docs are currently on this README on GitHub
Technologies	Golang, Linux, Docker (not required for use), AWS S3.
Msg delivery semantics	Exactly once, exact in-order delivery within a stream.
Storage capacity	Practically unlimited. You have to pay your bills though. :)
Data durability	All writes transactionally backed by Write-Ahead-Log just like in databases.
High availability	Planned, going to use Hashicorp's Raft implementation.
Data stored at	AWS S3. Google Storage support planned.
Encryption at transport	TLS (CA & server certs automatically managed)
Encryption at rest	AES256-CTR. Encryption keys are not trusted to AWS.
Security	Our security policy & information

Docs

How-to's & tutorials:

Quickstart (for users)
Example app using Event Horizon
- Best way to grasp what Event Horizon is about, but you should also read quickstart.
- Feature highlight: how to live-migrate a under-heavy-writes service with zero downtime.
Building & contributing (for developers)
Operating

Architecture:

Meta:

Architecture summary

Writer manages writes to streams. Streams are divided into ~8 MB chunks which are stored compressed & encrypted in AWS S3, except the last, "live", chunk that we're writing into.

Data flow at a glance:

Somebody contacts Writer to append a line to the stream
Writer notifies pub/sub that a stream has new lines
All interested applications that want to read from any streams have a Pusher, which subscribes to pub/sub notifications for those streams.
Once the Pusher learns of new events, it issues a read to either:
- a Writer if it's live data OR
- S3 if it's older data.
Pusher then pushes this read to your application. Your application transactionally verifies that the read offset is at the offset of last stored write in database. If not, error is returned along with the correct offset, and Pusher continues to read from the correct offset.

Programming language support

To use EventHorizon somewhere, that language has to be able to receive events over HTTP + JSON, in just one HTTP path. There's even a helper library (pushlib - look at the architecture diagram) for that, and it's only a small amount of code.

Existing implementations of pushlib:

Go
- library managed in this main repo
- example application
node.js (coming soon)
PHP (coming soon)
Your programming language not yet supported?
- It's somewhat easy to implement - contributions appreciated :)

License & fair play

Legal license: Apache 2.0 - free software.

Fair play: if you benefit commercially from the use of this project in any significant capacity, any help would be appreciated (but not legally required):