moby--moby/libnetwork/docs/remote.md

Remote Drivers
==============

The `drivers.remote` package provides the integration point for dynamically-registered drivers. Unlike the other driver packages, it does not provide a single implementation of a driver; rather, it provides a proxy for remote driver processes, which are registered and communicate with LibNetwork via the Docker plugin package.

For the semantics of driver methods, which correspond to the protocol below, please see the [overall design](design.md).

## LibNetwork integration with the Docker `plugins` package

When LibNetwork initialises the `drivers.remote` package with the `Init()` function, it passes a `DriverCallback` as a parameter, which implements `RegisterDriver()`. The remote driver package uses this interface to register remote drivers with LibNetwork's `NetworkController`, by supplying it in a `plugins.Handle` callback.

The callback is invoked when a driver is loaded with the `plugins.Get` API call. How that comes about is out of scope here (but it might be, for instance, when that driver is mentioned by the user).

This design ensures that the details of driver registration mechanism are owned by the remote driver package, and it doesn't expose any of the driver layer to the North of LibNetwork.

## Implementation

The remote driver implementation uses a `plugins.Client` to communicate with the remote driver process. The `driverapi.Driver` methods are implemented as RPCs over the plugin client.

The payloads of these RPCs are mostly direct translations into JSON of the arguments given to the method. There are some exceptions to account for the use of the interfaces `EndpointInfo` and `JoinInfo`, and data types that do not serialise to JSON well (e.g., `net.IPNet`). The protocol is detailed below under "Protocol".

## Usage

A remote driver proxy follows all the rules of any other in-built driver and has exactly the same `Driver` interface exposed. LibNetwork will also support driver-specific `options` and user-supplied `labels` which may influence the behaviour of a remote driver process.

## Protocol

The remote driver protocol is a set of RPCs, issued as HTTP POSTs with JSON payloads. The proxy issues requests, and the remote driver process is expected to respond usually with a JSON payload of its own, although in some cases these are empty maps.

### Errors

If the remote process cannot decode, or otherwise detects a syntactic problem with the HTTP request or payload, it must respond with an HTTP error status (4xx or 5xx).

If the remote process can decode the request, but cannot complete the operation, it must send a response in the form

    {
        "Err": string
    }

The string value supplied may appear in logs, so should not include confidential information.

### Handshake

When loaded, a remote driver process receives an HTTP POST on the URL `/Plugin.Activate` with no payload. It must respond with a manifest of the form

    {
        "Implements": ["NetworkDriver"]
    }

Other entries in the list value are allowed; `"NetworkDriver"` indicates that the plugin should be registered with LibNetwork as a driver.

### Create network

When the proxy is asked to create a network, the remote process shall receive a POST to the URL `/NetworkDriver.CreateNetwork` of the form

    {
        "NetworkID": string,
        "Options": {
            ...
        }
    }

The `NetworkID` value is generated by LibNetwork. The `Options` value is the arbitrary map given to the proxy by LibNetwork.

The response indicating success is empty:

    `{}`

### Delete network

When a network owned by the remote driver is deleted, the remote process shall receive a POST to the URL `/NetworkDriver.DeleteNetwork` of the form

    {
        "NetworkID": string
    }

The success response is empty:

    {}

### Create endpoint

When the proxy is asked to create an endpoint, the remote process shall receive a POST to the URL `/NetworkDriver.CreateEndpoint` of the form

    {
        "NetworkID": string,
        "EndpointID": string,
        "Options": {
            ...
        },
        "Interfaces": [{
            "ID": int,
            "Address": string,
            "AddressIPv6": string,
            "MacAddress": string
        }, ...]
    }

The `NetworkID` is the generated identifier for the network to which the endpoint belongs; the `EndpointID` is a generated identifier for the endpoint.

`Options` is an arbitrary map as supplied to the proxy.

The `Interfaces` value is a list with values of the form given. The fields in the `Interfaces` entries may be empty; and the `Interfaces` list itself may be empty. If supplied, `Address` is an IPv4 address and subnet in CIDR notation; e.g., `"192.168.34.12/16"`. If supplied, `AddressIPv6` is an IPv6 address and subnet in CIDR notation. `MacAddress` is a MAC address as a string; e.g., `"6e:75:32:60:44:c9"`.

A success response is of the form

    {
        "Interfaces": [{
            "ID": int,
            "Address": string,
            "AddressIPv6": string,
            "MacAddress": string
        }, ...]
    }

with values in the `Interfaces` entries as above. For each entry, an `ID` and `MacAddress` and either or both of `Address` and `AddressIPv6` must be given. The `ID` is arbitrary but must differ among entries. It is used to identify, within the scope of the endpoint, an individual interface during a `Join` call.

If the remote process was supplied entries in `Interfaces`, it must respond with an empty `Interfaces` list. LibNetwork will treat it as an error if it supplies a non-empty list and receives a non-empty list back, and roll back the operation.

### Endpoint operational info

The proxy may be asked for "operational info" on an endpoint. When this happens, the remote process shall receive a POST to `/NetworkDriver.EndpointOperInfo` of the form

    {
        "NetworkID": string,
        "EndpointID": string
    }

where `NetworkID` and `EndpointID` have meanings as above. It must send a response of the form

    {
        "Value": { ... }
    }

where the value of the `Value` field is an arbitrary (possibly empty) map.

### Delete endpoint

When an endpoint is deleted, the remote process shall receive a POST to the URL `/NetworkDriver.DeleteEndpoint` with a body of the form

    {
        "NetworkID": string,
        "EndpointID": string
    }

where `NetworkID` and `EndpointID` have meanings as above. A success response is empty:

    {}

### Join

When a sandbox is given an endpoint, the remote process shall receive a POST to the URL `NetworkDriver.Join` of the form

    {
        "NetworkID": string,
        "EndpointID": string,
        "SandboxKey": string,
        "Options": { ... }
    }

The `NetworkID` and `EndpointID` have meanings as above. The `SandboxKey` identifies the sandbox. `Options` is an arbitrary map as supplied to the proxy.

The response must have the form

    {
        "InterfaceNames": [{
            SrcName: string,
            DstPrefix: string
        }, ...],
        "Gateway": string,
        "GatewayIPv6": string,
        "StaticRoutes": [{
            "Destination": string,
            "RouteType": int,
            "NextHop": string,
            "InterfaceID": int
        }, ...]
        "HostsPath": string,
        "ResolvConfPath": string
    }

`Gateway` is optional and if supplied is an IP address as a string; e.g., `"192.168.0.1"`. `GatewayIPv6` is optional and if supplied is an IPv6 address as a string; e.g., `"fe80::7809:baff:fec6:7744"`. `HostsPath` is optional, as is `ResolvConfPath`.

The entries in `InterfaceNames` represent veths that should be moved by LibNetwork into the sandbox; the `SrcName` is the name of the veth that the remote process created, and the `DstPrefix` is a prefix for the name the veth should have after it has been moved into the sandbox (LibNetwork will append an index to make sure the actual name does not collide with others).

The position of the entries in the list must correspond to the interface IDs given in the response to `/NetworkDriver.CreateEndpoint` as described above. For example, if there were two `Interfaces` in the create endpoint response, with IDs `0` and `1`, then the `InterfaceNames` list would have the interface names respectively in positions `0` and `1`of the list. (For this reason it is recommended that interfaces are given sequential IDs starting with `0`.)

The entries in `"StaticRoutes"` represent routes that should be added to an interface once it has been moved into the sandbox. Since there may be zero or more routes for an interface, unlike the interface names they can be supplied in any order, and are marked with the `InterfaceID` of the corresponding interface.

Routes are either given a `RouteType` of `0` and a value for `NextHop`; or, a `RouteType` of `1` and no value for `NextHop`, meaning a connected route.

### Leave

If the proxy is asked to remove an endpoint from a sandbox, the remote process shall receive a POST to the URL `/NetworkDriver.Leave` of the form

    {
        "NetworkID": string,
        "EndpointID": string
    }

where `NetworkID` and `EndpointID` have meanings as above. The success response is empty:

    {}