1. (MODAClouds) Load-balancer

This document is a draft!

Contents

(MODAClouds) Load-balancer

1.1. Concepts

Load-balancer

A HTTP (or TCP) proxy service that "receives" "transaction" "requests" from "clients" "outside" the "system", then "validates" and "forwards" these "transaction" "requests" to a "target" inside a "pool" based on a pre-determined "policy", following that waiting for the corresponding "transaction" "reply" which is "forwarded" back to the original "client".

by "transactions", "requests" and "responses" we understand individual HTTP request / reply exchanges --- i.e. one HTTP request and its counterpart response, as opposed to HTTP sessions that span over multiple HTTP transactions --- or in case of TCP an individual connection; we don't call it "connection" because in the case of HTTP multiple transactions can be multiplexed inside the same connection;
by "client" we understand any remote process, be it an actual client software, or another service;
by "system" we understand the entire application hosted in the cloud;
by "outside" we understand either coming from clients not running on the application's VM's; (it should be noted that nothing forbids any of the application components to access the load-balancer;)
by "validates" we understand at most syntax validation, and some basic semantic validation, but not things like authentication or authorization; (it should be noted that SSL-termination could be done by the load-balancer itself;)

Gateway

A load-balancer is able to proxy multiple types of services at the same time, without the requests interfering; e.g. in case an application exposes both a HTTP service (or more) but also an SMTP or XMPP service, then the same load-balancer could take handle both types of services. There is a one-to-one relation between a gateway and an exposed service, i.e. one gateway for the HTTP service, and another for SMTP, etc. Evidently a load-balancer can have more than one gateways, thus a many-to-one relation with the load-balancer.

Endpoint

A logical entity identified by a network address, i.e. an IP address and port in case of TCP.

Gateway-endpoint

The network address where the load-balancer should expect to receive requests for a particular gateway. There is a many-to-one relation between the endpoint and a gateway, i.e. the same service could be accessible by different IP's or ports (such a case is not usually found in the cloud environment).

Pool

The counterpart of a gateway, representing a set of internal services that actually handle the requests. There is a many-to-many relation between the gateways and the pools, potentially based on some "routing-rules". A valid use-case for two pools that serve the same gateway is A-B testing with a new release of a service, i.e. the newly deployed pool of services should handle a small fraction of the requests.

Target

The member of a pool, which represents the individual internal service that actually handles the request. There is a many-to-one relation with pools, i.e. one target can belong to only one pool (although nothing forbids the operator to register the same target endpoint in another pool). The pool is characterized by a single endpoint, a set of classes, and a number of maximum outstanding transactions.

Target-class

Nothing more than a label which can be used in load-balancing policies.

Policy

Describes the used algorithm and its parameters that should govern the dispatching of requests between the a pool's targets. Inside the parameters, a policy could individually single-out a target, or (even better) it could differentiate them them by using target classes.

Controller

A small service exposing a REST-ful HTTP API, that allows the operator (through a set of dedicated tools) or other sub-systems (like the self-adaptation policy), to dynamically change the configuration of the load-balancer.

1.2. Policies

Note: In any of policies described below, the request is interpreted as an individual HTTP transaction, irrespective of the logical HTTP session it belongs to. Thus none of these policies are "sticky", i.e. send all requests originating from the same client to the same server.

1.2.1. Round-robin

(For details search inside #HAProxy-Configuration for the option balance and the algorithm round-robin.)

Probably the most useful policy, which routes requests in a circular manner where each target could have different "span" on the "circle".

Algorithm parameters:

weights: a map stating the weight of each target, class of targets, or default value; it is represented by a value between 0 (inclusive) and 1 (inclusive), where 0 means totally excluded from the routing decision; the actual weights are computed by first assigning values to all of the pool targets, then normalizing the values, with a precision that gives at least 256 levels;

1.2.2. Least-connections

(For details search inside #HAProxy-Configuration for the option balance and the algorithm leastconn.)

Directs requests to those targets which have the least number of outstanding transactions.

Note: It is unclear for now how the weights interact with the algorithm (the documentation is "shy" in this respect).

Algorithm parameters:

weights: the same like in the case of round-robin;

1.2.3. Backfill

(For details search inside #HAProxy-Configuration for the option balance and the algorithm first.)

Useful in cases where there is a direct link between the number of outstanding transactions (or connections in TCP mode) and the load on the target, or when coupled with targets that when not handling requests are able to get into a "stand-by" mode. (Classical examples are database connections, where each connection has a clear overhead in terms of memory, or CPU-intensive HTTP requests.) Internally the load-balancer uses the maximum outstanding transactions number as the limiting factor.

Algorithm parameters:

priorities: similar with weights in the previous cases, except that the values are not normalized; in case of equal priorities a random order is applied at configuration time (i.e. the filling order is always exactly the same;)

1.3. References

HAProxy Configuration Manual

: http://haproxy.1wt.eu/download/1.5/doc/configuration.txt

1.4. API

1.4.1. Operations

1.4.1.1. General considerations

In all the circumstances below the following rules apply (except where explicitly noted):

all HTTP requests must have proper values for the following headers:
- Content-Type, usually set to application/json, for POST and PUT methods;
- (optionally) Content-Encoding set to identity; (i.e. no compression is allowed;)
- Content-Length set to identify the size of the request body (if not chunked);
- (optionally) Transfer-Encoding set to chunked, in case the request body is "streamed";
- Accept used to designate which document formats are understood by the requester; the only format that is guaranteed to be available on the server is JSON, but for some requests XML or variants of HTML could be supported;
- other headers like could be ignored or a failure could be signaled (status codes 4xx or 5xx);
all HTTP responses will have proper values for the following headers:
- Content-Type, usually set to application/json, which should be in-line with the Accept request header; (otherwise an error shall be signaled, status codes 4xx or 5xx);
- Content-Encoding set to identity; (i.e. no compression is supported;)
- Content-Length set to a proper value (if not chunked);
- (optionally) Transfer-Encoding, like above;
in case of PUT requests, the entity descriptor completely replaces the previous existing descriptor;
a PUT request can also lead to removal of existing linked entities in case their identifier or alias doesn't appear in the new list / map; (i.e. for a gateway the endpoints attribute is a map of existing endpoints; if on PUT an alias is absent from that map, that endpoint is automatically deleted;) moreover renaming aliases is possible by changing the key in the map and keeping the associated identifier value; however new linked entities can not be created this way;

1.4.1.2. Gateways

`GET` `/v1/gateways`: a list of identifiers for the defined gateways; (i.e. the response body is a JSON list containing strings;)
`GET` `/v1/gateways/{gateway}`: obtain the definition document for a particular gateway (identified by the token {gateway}); the response body is a <gateway-descriptor>;
`PUT` `/v1/gateways/{gateway}`: create a new gateway (identified by the token {gateway}), or update an existing gateway in case one exists with the given identifier; the request body is a <gateway-descriptor>;
`DELETE` `/v1/gateway/{gateway}`: deletes the designated gateway;
`GET` `/v1/gateways/{gateway}/endpoints`: a list of identifiers for all the gateway's endpoints;
`GET` `/v1/gateways/{gateway}/endpoints/{endpoint}`: (evident)
`PUT` `/v1/gateways/{gateway}/endpoints/{endpoint}`: (evident)
`DELETE` `/v1/gateways/{gateway}/endpoints/{endpoint}`: (evident)
`GET` `/v1/gateways/{gateway}/pools`: a list of aliases for all the associated pools;
`GET` `/v1/gateways/{gateway}/pools/{pool}`: returns the identifier (the one to be used in /v1/pools/{pool}) of the designated pool; (i.e. in the first URL {pool} is the alias, the key in the gateway's pools object;)
`PUT` `/v1/gateways/{gateway}/pools/{pool}`: (evident)
`DELETE` `/v1/gateways/{gateway}/pools/{pool}`: (evident)

1.4.1.3. Pools

`GET` `/v1/pools`: (evident)
`GET` `/v1/pools/{pool}`: (evident)
`PUT` `/v1/pools/{pool}`: (evident)
`DELETE` `/v1/pools/{pool}`: (evident)
`GET` `/v1/pools/{pool}/targets`: (evident)
`GET` `/v1/pools/{pool}/targets/{target}`: (evident)
`PUT` `/v1/pools/{pool}/targets/{target}`: (evident)
`DELETE` `/v1/pools/{pool}/targets/{target}`: (evident)

1.4.1.4. Policies

`GET` `/v1/pools/{pool}/policy`: obtain the policy of the designated pool; the response body is either the JSON null value in case no policy exists, or a <policy>;
`PUT` `/v1/pools/{pool}/policy`: updates the policy of the designated pool; (by using the JSON null value the same effect as DELETE is obtained;)
`DELETE` `/v1/pools/{pool}/policy`: reset the policy of the designated pool to a default value;

1.4.1.5. Controller

`GET` `/v1/controller/_export-cdb`: export a CDB variant of the configuration database (to be used by the HAProxy configuration tool);
`GET` `/v1/controller/_export-sql`: export a SQLite snapshot of the configuration database (via the .dump command);
`PUT` `/v1/controller/_import-sql`: import a SQLite snapshot of the configuration database, removing all previous content;
`POST` `/v1/controller/commit`: generate the load-balancer configuration, then signal the reload; it expects the JSON null value as the request (allowing future changes), and returns a <generic-operation-return> response;

1.4.2. Descriptors

1.4.2.1. Gateways

<gateway-descriptor> ::= {
    "protocol" : <gateway-protocol>,
    "endpoints" : {
        <alias> : <entity-identifier> *,
    },
    "pools" : {
        <alias> : <entity-identifier> *,
    },
    "enabled" : <boolean>,
    @<annotation-attribute> ?,
    @<standalone-entity-attributes>,
}

<gateway-protocol> ::= ("http" | "tcp")

<gateway-endpoint> ::= {
    "address" : <endpoint-address>,
    @<annotation-attribute> ?,
    @<standalone-entity-attributes>,
}

1.4.2.2. Pools

<pool-descriptor> ::= {
    "targets" : {
        <alias> : <entity-identifier> *,
    },
    "enabled" : <boolean>,
    @<annotation-attribute> ?,
    @<standalone-entity-attributes>,
}

<target-descriptor> ::= {
    "endpoint" : {
        "address" : <endpoint-address>,
    }
    "enabled" : <boolean>,
    "classes" : [ <target-class> *, ],
    "maximum-outstanding-transactions" : (<positive-integer> | 0),
    @<annotation-attribute> ?,
    @<standalone-entity-attributes>,
}

// FIXME: Provide a regular expression!
<target-class> ::= <string>

1.4.2.3. Policies

<policy-descriptor> ::= (
        <round-robin-policy-descriptor> |
        <least-connections-policy-descriptor> |
        <backfill-policy-descriptor>
)

<round-robin-policy-descriptor> ::= {
    "weights" : {
        <policy-selector> : <float(0,1)> *,
    }
    @<annotation-attribute> ?,
    @<standalone-entity-attributes>,
}

<least-connections-policy-descriptor> ::= {
    "weights" : {
        <policy-selector> : <float(0,1)> *,
    }
    @<annotation-attribute> ?,
    @<standalone-entity-attributes>,
}

<backfill-policy-descriptor> ::= {
    "priority" : {
        <policy-selector> : <float(0,1)> *,
    }
    @<annotation-attribute> ?,
    @<standalone-entity-attributes>,
}

// FIXME: Provide regular expressions!
<policy-selector> ::= (<policy-target-selector> | <policy-class-selector> | <policy-default-selector>)
<policy-target-selector> ::= ...
<policy-class-selector> ::= ...
<policy-default-selector> ::= ...

1.4.2.4. Generic

// FIXME: Provide a regular expression!
<alias> ::= <string>

<endpoint-address> ::= ("tcp:" + <ip-address> + ":" + <ip-port>)

// FIXME: Provide regular expressions!
<ip-address> ::= <ipv4-address> | <ipv6-address>
<ipv4-address> ::= ...
<ipv6-address> ::= ...

<ip-port> ::= <integer(1, 65535)>

<selfdescribing-entity-attributes> ::= {
    "_schema" : <schema-identifier>,
    "_schema/version" : <schema-version>,
}

<standalone-entity-attributes> ::= {
    @<selfdescribing-entity-attributes>,
    "_identifier" : <entity-identifier>,
    "_revision" : <entity-revision>,
    "_revision/previous" : (<entity-revision> | null)?,
    "_revision/originator" : (<url> | null)?,
}

// FIXME: Provide regular expressions!
<schema-identifier> ::= ...
<schema-version> ::= ...
<entity-identifier> ::= ...
<entity-revision> ::= ...

// FIXME: Provide a regular expression!
<url> ::= ...

1.5. Databases

1.5.1. SQLite

* documents:
 * identifier: string;
 * revision: string;
 * schema: string;
 * schema_version: string;
 * data: string (JSON);

1.5.2. CDB

...