(SPECS) Heka

Important

This document is a draft!

Important

This document was created in March 2014, thus some of the information within could be out-of-date!

1   Concepts

  • inputs / outputs / circular-buffer;
  • filters / sandboxed-filters;
  • decoders;

2   Notes

Advantages

  • modular and distributed architecture; [Heka-01]
  • one-process design --- i.e., all modules live inside the same process, thus no fork-ing or exec-ing happens at run-time;
  • native-compiled executable --- i.e., no language run-time requirements;
  • written in a portable (and compilable) language, [Go];
  • support for "sandboxed filters" which can be written in [Lua]_;
  • support for message routing; [1]
  • seems to support both monitoring and logging; [1]
  • can receive and send monitoring data via: [statsd]_ protocol, [syslog]_ protocol, [AMQP], HTTP, TCP or UDP, specialized process; [Heka-04] [1]
  • seems to support both push and pull data input; [1];
  • can write "whisper" database format ready for [Graphite]_;
  • can read log files for monitoring data; [Heka-04]
  • the project is backed by Mozilla;
  • the project is very lively according to the GitHub statistics: ~2200 commits, ~30 contributors; [Heka-Git]
  • the project has an active mailing list, although not too many posts per month (~10-20); [Heka-ML]
  • some client libraries are available in Python and NodeJS for sending monitoring data; [Heka-01]

Disadvantages

  • adding new module types requires recompilation from source; (however management is done through configuration files as expected and doesn't require recompilation;)
  • due to one-process, a badly devised module can interfere with the other modules, although there is explicit support for handling such situations (see [Heka-03], section "Configuring Restarting Behavior");
  • limited storage options; [1]
  • there isn't a production-ready version, although there have been about 5 releases;
  • the documentation is rather sparse;

Configuration

  • either through a single file or a folder (all files are merged a-la *.d); [Heka-03]
  • uses the [TOML] format (simple INI-style files); [Heka-03]

Input / output

...

Data formats

...

Routing

...

Licensing

[1](1, 2, 3, 4, 5) This paragraph needs to be checked!

Docutils System Messages

System Message: ERROR/3 (<string>, line 34); backlink

Unknown target name: "lua".

System Message: ERROR/3 (<string>, line 37); backlink

Unknown target name: "statsd".

System Message: ERROR/3 (<string>, line 37); backlink

Unknown target name: "syslog".

System Message: ERROR/3 (<string>, line 39); backlink

Unknown target name: "graphite".