# README
tracelistener
UNIX named pipes-based real-time state listener for Cosmos SDK blockchains.
See also the demeris-backend docs for an overview of the architecture.
Description
What it is
Tracelistener is a program that reads the Cosmos SDK store
in real-time and dumps the result in a relational database, essentially creating a 1:1 copy of the data available in a module's prefix store.
The relational database of choice is CockroachDB — a Postgres protocol-compatible relational database — while the entirety of tracelistener is written in Go.
Tracelistener is a vital component of the Emeris backend, since it provides
- account balances
- staking amounts
- IBC channels, connections, clients informations
without us having to query those information from full-nodes.
By not querying full-nodes, tracelistener reduces nodes load and diminishes the chance of load-related issues — like nodes not receiving/parsing blocks due to high query amount.
Given the tightly-coupled nature of tracelistener to a Cosmos SDK node, they must be executed together on the same machine.
How it works
The Cosmos SDK has a little-known feature called store tracing, which tracks each and every store operation on a file.
Cosmos SDK store defines four kind of store operations:
write
delete
read
iterRange
Tracelistener is only concerned with the first two.
Each store operation is divided by a newline, and the store operation itself is serialized as JSON.
To reduce hard drive load on the hardware node which is running, tracelistener opens a UNIX named pipe (commonly referred to as FIFO) on which the Cosmos SDK node will then write store tracing lines.
A UNIX shell proof of concept can be summarized like this:
# This example needs to be executed in two separate terminals.
# Terminal 1
mkfifo /tmp/tracelistener.fifo
gaiad start --trace-store /tmp/tracelistener.fifo
# Terminal 2
cat /tmp/tracelistener.fifo
In the first terminal we create a named pipe in /tmp/tracelistener.fifo
, and then start gaiad
with the --trace-store
.
gaiad
will look like it's stuck on the Tendermint initialization phase: it's normal, FIFO's block writes until there's a reader.
In the second Terminal the cat
command starts printing JSON store tracing lines, and gaiad
will unblock itself and resume execution.
If cat
is killed before gaiad
, the latter will experience a consensus failure: this is normal, and happens because it is not possible for a program to write on a closed pipe.
In a production environment, tracelistener must always be executed before the SDK node, and killed last.
For each JSON line read, tracelistener unmarshals it into a Go struct and proceeds with the parsing routine — we will refer to this object as trace operation from now on.
A trace operation is defined as follows:
type TraceOperation struct {
Operation string `json:"operation"`
Key []byte `json:"key"`
Value []byte `json:"value"`
BlockHeight uint64 `json:"block_height"`
TxHash string `json:"tx_hash"`
}
In tracelistener, a processor is an entity that is capable of handling SDK store rows.
Right now there's only one processor, called gaia*.***
Each processor contains modules, which are entities capable of
- understanding what's inside a trace operation
- unmarshal the protobuf bytes contained in
Value
- return a database object and
INSERT
statement to be executed
To understand where to route each trace operation, processors look at the prefix bytes on each operation Key
.
Each module is responsible of validating a trace operation against a well-defined set of rules, because Key
prefixes could be shared among different Cosmos SDK modules — for example, the 0x02
prefix is used by the IBC channels module as well as the supply
one, so the IBC channels module must be sure to not write supply
database rows in its table.
Once a trace operation has been processed, it is then sent over for database execution.
Database schema is automatically migrated each time tracelistener is executed, but this behavior will change in the future.
Dependencies
- CockroachDB
- your Cosmos SDK-based blockchain node
Configuration
tracelistener can be configured either through a configuration file or through environment variables.
The configuration file must be named tracelistener.toml
and must live in either:
/etc/tracelister/tracelistener.toml
$HOME/.tracelistener/tracelistener.toml
./tracelistener.toml
Every configuration entry can be accessed through an environment variable with the same name all uppercase,
prefixed with the TRACELISTENER_
string.
While the configuration file field names are case-insensitive, environment variables are case-sensitive.
Configuration value | Default value | Required | Meaning |
---|---|---|---|
FIFOPath | .tracelistener.fifo | no | UNIX named pipe path where tracelistener will read data from |
DatabaseConnectionURL | yes | Database connection URL used to connect to CockroachDB | |
LogPath | ./tracelistener.log | no | Path where tracelistener will write its log file |
Type | yes | Type of data processor used by tracelistener to process data it reads from FIFOPath | |
Debug | false | no | Enable debug logs, disable file logging |
Type-specific configuration
Gaia configuration
This configuration is used when Type
is gaia
.
Configuration value | Default value | Required | Meaning |
---|---|---|---|
ProcessorsEnabled | bank | no | List of module processors to be enabled, and which will process data coming from FIFOPath |
The list of processors for gaia is the following:
bank
ibc
liquidityPool
liquiditySwaps
Running tracelistener
-
run a CockroachDB instance somewhere
-
write a configuration file or set the environment appropriately.
-
build it:
go build -v --ldflags="-s -w" github.com/emerishq/tracelistener/cmd/tracelistener
-
run it:
./tracelistener
-
run your chain with the
--trace-store
parameter# in this instance, tracelistener FIFO path is set to /home/tl/tracelistener.fifo gaiad start --trace-store=/home/tl/tracelistener.fifo
Docker container
This repository contains a Docker image which can be used to run a tracelistener container.
Build it with:
docker build -t tracelistener:latest --build-arg GIT_TOKEN={YOUR-TOKEN} .
Dependencies & Licenses
The list of non-{Cosmos, AiB, Tendermint} dependencies and their licenses are:
Module | License |
---|---|
containerd/fifo | Apache 2.0 |
go.uber.org/zap | MIT |
gorilla/websocket | BSD-2 |
cockroachdb/cockroach-go | Apache 2.0 |
stretchr/testify | MIT |
gogo/protobuf | Only on redistr. |
go-playground/validator | MIT |
nxadm/tail | MIT |
iamolegga/enviper | MIT |
spf13/viper | MIT |
jackc/pgx | MIT |
jmoiron/sqlx | MIT |
gin-gonic/gin | MIT |
natefinch/lumberjack | MIT |
lib/pq | Unrestricted |
ethereum/go-ethereum | GNU LGPL |