(mOSAIC) Configuration

Requirements

• human readable / writable, ASCII based;
• easily parsable in most languages; (e.g. property files, INI, JSON, S-expressions, YAML, XML;)
• presenting a "certain" level of typing information (i.e. syntactically discerning between numbers, booleans, strings, etc); (e.g. JSON, S-expressions, YAML, maybe XML with XML Schema;)
• easily exposing / enforcing certain modeling constructs; (i.e. allowing to discern between properties of an entity, and properties of aggregated entities contained in a parent entity -- as in connector properties used in a cloudlet;) (e.g. XML with XML Schema;)
• self describable: (i.e. having only the configuration, and knowing the generic rules, being able to display / process given entities properties and aggregated entities;)
• composable / stacking options; (i.e. having a base ("default") configuration, and on-top layering a configuration specific to a certain "packaging" of the component, and on-top of than again layering a configuration specific to a certain component instance;)
• presenting "certain" level of "programmability" / "automatism"; (i.e. at least supporting variable definitions and replacement, could also include generative behaviour;)

Proposal

There are three (and one supplementary) proposed languages -- all based on JSON (or JSON equivalent like YAML) thus called "formats":

• configuration data format; (i.e. the one used by components to configure themselves);
• configuration description format; (i.e. the one used by the developer / user to "configure" the component, and based on which a configuration data instance is generated);
• configuration schema format; (i.e. the one used by the developer to "define" and "constrain" the semantic of the final data;)
• configuration WUI schema format; (i.e. used to generate web user interfaces, for creating and manipulating configuration data;)

Configuration data format

BNF systax:

• <property-identifier-component> ::= /^[a-z0-9]([a-z0-9-]*[a-z0-9])$/
  <property-identifier> ::= <property-identifier-component> ( "." <property-identifier-component> ) *
  <directive-identifier> ::= /^\@[a-z0-9]([a-z0-9-]*[a-z0-9])$/
  
  <configuration> ::= {
      "@schema" : {
          "version" : "1.0",
      } | "1.0",
      "@type" : "entity",
      <entity>, // i.e. inherit all members of <entity>
  }
  
  <entity> ::= {
      "@type" : "entity",
      <property-identifier> : <property-value> | <property-namespace>,
      ...
  }
  
  <property-namespace> ::= {
      "@type" : "namespace",
      <property-identifier> : <property-value> | <property-namespace>,
      ...
  }
  
  <property-value> ::= <entity> | <map> | <json-value>
  
  <map> ::= {
      "@type" : "map",
      <json-map-key> : <property-value>,
      ...
  }
  
  <json-value> ::= <json-atom> | <json-array> | <json-map>
  <json-atom> ::= <json-integer> | <json-real> | <json-string> | <json-boolean> | <json-null>
  <json-array> ::= [ <json-value>, ... ]
  <json-map> ::= {
      "@type" : "json",
      <json-map-nested>,
  }
  <json-map-nested> ::= {
      <json-map-key> : <json-value> | <json-map-nested>,
      ...
  }
  <json-map-key> ::= "" | /^[^@`].*$/ | /^\\[@`].*$/
  <json-string> ::= "" | /^[^`].*$/ | /^\\^.*$/

Example:

• configuring a "composer" component, part of Gisheo application:
  • configuration data:

    • {
          "@type" : "entity",
          "@schema" : "1.0",
          "container" : {
              "@type" : "entity",
              "threads" : {
                  "@type" : "namespace",
                  "min" : 1,
                  "max" : 4,
                  "priority" : "high"
              },
              "logging.levels" : {
                  "@type" : "json",
                  "" : "warning",
                  "eu.mosaic.core" : "information",
                  "eu.mosaic.connectors" : "warring"
              }
          },
          "cloudlet" : {
              "@type" : "entity",
              "class" : "eu.gisheo.tiles.ComposerCloudlet",
              "configuration" : {
                  "@type" : "entity",
                  "algorithm" : "linear"
              },
              "connectors" : {
                  "@type" : "map",
                  "store" : {
                      "@type" : "entity",
                      "type" : "kv",
                      "driver.group" : "57892710-7d1b-4fd6-b757-3a2015d0f21d",
                      "driver.timeout" : 60.0
                  },
                  "requests" : {
                      "@type" : "entity",
                      "type" : "mq",
                      "driver" : {
                          "@type" : "namespace",
                          "group" : "6a930c3e-a622-4270-9631-dc8586c5f1b9",
                          "timeout" : 60.0
                      }
                  }
              }
          }
      }

  • interpretation (entities with properties):
    • the root, i.e. the whole component configuration:

      • container: a property bound to the entity representing a container; (the entity doesn't have a name;)

      • cloudlet: as above bound to an anonymous entity;

    • the anonymous container (bound to the container property of the root entity):

      • threads.min: 1;

      • threads.max: 4;

      • threads.priority: high;

      • logging.levels: an opaque JSON object  {"":"warning", "eu.mosaic.core" : "information", "eu.mosaic.connectors" : "warning"} ;

    • the anonymous cloudlet (bound to the cloudlet property of the root entity):

      • class : eu.gisheo.tiles.ComposerCloudlet;

      • configuration: a special "configuration" entity specific to the cloudlet API and the given cloudlet;

      • connectors: a map of entities representing connector configurations;

    • the store connector (bound to the store key of the map bound to the connectors property of the cloudlet):

      • type: kv, i.e. the connector type, and not the configuration "node" type;

      • driver.group: ...;

      • driver.timeout: 60.0;

    • the requests connector (bound to the requests key of the map bound to the connectors property of the cloudlet):

      • type: mq, as in the case of store see the difference between the keys @type and type;

      • driver.group: ...;

      • driver.timeout: 60.0;

  • observations:
    • see how properties can be hierarchical (or nested, or prefixed with namespaces) in two different ways:

      • writing the fully qualified name (i.e. driver.group) for each property;

      • or nesting them in an "@type" : "namespace" JSON object; (just like the namespace keyword in C++;)

Configuration description format

variable-identifier ::= /^\$[a-z0-9]([a-z0-9-]*[a-z0-9])$/

...

References

• JSON Patch;

• An exhaustive specification of the GYP input file format;

• Configuring Python applications with the config module;

• Python ConfigParser -- Configuration file parser;

• Erlang reference manual -- config;

• Java API -- Properties;

• 5/ZDCF - ZeroMQ Device Configuration File;

• 4/ZPL -- ZeroMQ Property Language;

• JSON;

• YAML;

• Clojure: DSLs as Configuration;

• The Art of Unix Programming -- Configuration;

• Cloanto Implementation of INI File Format;

• Boost.PropertyTree;

• ConfigObj 4 Introduction and Reference;

• indexzero / nconf;

• MongoDB -- Modifier Operations;

• Apache Commons -- Java Configuration API;

• libConfuse;