================================= cedar -- Build Data Warehouse

Overview

Cedar is a collection of data-focused tools for data generated by and used in Evergreen builds. It is not a part of Evergreen, but serves as a companion service and tool kit. In particular, Cedar focuses on the "offline" data processing cases, and providing access to data to support external user interfaces (e.g. cli tools, data visualizations, etc.)

Build operations generate data, and Evergreen stores most of this data directly (e.g. the outcome of tasks, artifacts produced by builds). There's other data, specific to particular projects, that is less naturally structured or less specifically linked to specific execution events. Additionally, Cedar provides a platform to collect and analyze data about an Evergreen deployment in the large.

Components

- A simple JSON-only REST web service for data access (see the ``rest`` package).

- A gRPC service for data storage (see the ``rpc`` package).

- A command line tool that provides a REST client for common operations, as well as independent data collection and
  exploration tools (see the ``operations`` package).

- An offline task processing framework based on `amboy <https://github.com/mongodb/amboy/>`_ for application-level data
  aggregation operations (see the ``units`` package).

Data Types ~~~~~~~~~~

- Logs for Evergreen tasks.

- Test result logs and metadata for Evergreen tasks.

- A simple log storage system, as a proof of concept. Logs stream from the client to the Cedar application which stores
  metadata about the logs and then saves data to S3. Out of band, jobs can process log data.

Use ---

#. Download and compile Cedar. I prefer something like: ::

     git clone https://github.com/evergreen-ci/cedar
     cd cedar
     make cedar

#. Explore the help menus in the ``cedar`` binary: ::

     cedar --help
     cedar service --help
     cedar client --help
     cedar worker --help

#. Enjoy!

Development -----------

The Cedar project uses a ``makefile`` to coordinate testing. Use the following command to build the Cedar binary: ::

  make cedar

The artifact is at ``build/cedar``. The makefile provides the following targets:

``test`` Runs all tests, sequentially, for all packages. Some of the tests require a *replica set* to be running under http://127.0.0.1:27017.

``test-<package>`` Runs all tests for a specific package

``lint``, ``lint-<package>`` Installs and runs the ``gometaliter`` with appropriate settings to lint the project.