# README

DarkMetrix Log

A fast network logging system powered by Apache Kafka.

Architecture

Quick start

Preperation

An Apache Kafka need to be installed previously and create a topic.

./kafka-topics.sh --zookeeper localhost:2181 --create --replication-factor 1 --partitions 10 --topic net_log

See DarkMetrix/log/agent or DarkMetrix/log/server to get more information about Kafka setting.

Install from source

git clone [email protected]:DarkMetrix/log.git

Build agent & run (You will need a root user)

$cd DarkMetrix/log/agent/src
$go build -o ../bin/dm_log_agent
$../admin/start.sh

Build server & run

$cd DarkMetrix/log/server/src
$go build -o ../bin/dm_log_server
$../admin/start.sh

Protocol

We use google/protobuf as the protocol, so you shuold use protobuf whatever programming language you use to marshal the log and send via unix domain socket.

log.proto

syntax = "proto2";

package log_proto;

//Log package
message LogPackage
{
    optional string project = 1;    //Project name(Also used as the folder name)
    optional string service = 2;    //Service name(Also used in the log file name)
    optional uint32 level = 3;      //Log level(INFO = 0, WARNING = 1, ERROR = 2, FATAL = 3)
    optional bytes  log = 4;        //Log content
};