Treatment distant

work/distant::distant

Configuration

⬡ distant_engine: work/distant::DistantEngine

Parameters

↳ var arch: Option<Arch> (work/resources/arch::Arch)
↳ var containers: Vec<Container> (work/resources::Container)
↳ var cpu: u32
↳ var edition: Option<string>
↳ var max_duration: u32
↳ var memory: u32
↳ var service_containers: Vec<ServiceContainer> (work/resources::ServiceContainer)
↳ var storage: u32
↳ var system: Option<System> = _ (work/resources/system::System)
↳ var tags: Vec<string>
↳ var volumes: Vec<Volume> (work/resources::Volume)

Inputs

⇥ trigger: Block<void>

Outputs

↦ access: Block<Access> (work/access::Access)
↦ errors: Stream<string>
↦ failed: Block<void>

Request for a distant worker.

Send a request to get a distant Mélodium worker, on which program distribution can be done.

• access is emitted once worker is accessible.
• failed is emitted if the worker request cannot be satisfied.
• errors stream the error messages that can occurs.

The request is based on given parameters:

• cpu: CPU amount requested for the worker, in millicores (1000 means one full CPU, 500 half of it);

• memory: memory requested for the worker, in megabytes;

• storage: filesystem storage requested for the worker, in megabytes;

• max_duration: maximum duration for which the worker will be effective, in seconds;

• arch: hardware architecture the worker must have (should be none if nothing specific is required);

• edition: edition of the Mélodium engine the worker must rely on (standard, desktop, container, …), can be none if nothing specific is required (any edition suits). It is not the container image, which is defined by the distribution requested in system; Compose runs the standard and container editions on an Alpine image, and edition only matters on container targets, workers on virtual machines satisfy any edition;

• system: what the worker must run on, either an operating system built with |linux, |windows, |macos, |freebsd and the other work/resources/system functions, or one exact machine image built with |image (a name resolved by whatever spawns the worker, never a container image); none if nothing specific is required, like for arch;

• containers: list of containers to instanciate alongside Mélodium engine as executors;

• service_containers: list of containers to instanciate alongside Mélodium engine as services;

• volumes: list of filesystem volumes that can be shared between the Mélodium engine and containers.

Every need except tags is a requirement: the worker is spawned satisfying it, or the request fails with a message naming the need. An operating system and a machine image cannot be requested together, and when both system and arch are given the image or system must exist for that architecture.

Tags are hints: they are never a requirement and never a reason to refuse a run. Whatever spawns the worker may use them, for example by copying them to the resources it creates.

Where the worker is spawned

The location of the dispatching engine tells where the worker is spawned:

Compose can only offer what its official image is. Each need is checked before anything is started:

A request Compose cannot satisfy starts nothing: failed is emitted and the errors say which need cannot be satisfied. The edition is the one of the engine the worker runs, while the container image is defined by the Linux distribution of system. Editions and systems are only requirements on container locations, a virtual machine worker satisfies any edition.

Example, asking Compose for a standard engine on Alpine:

use std/engine/util::startup
use std/engine/log::logErrors
use std/ops/option::|wrap
use work/distant::DistantEngine
use work/distant::distant
use work/resources/system::System
use work/resources/system::|linux_distribution

treatment main()
  model engine: DistantEngine(location="compose")
{
    startup()
    request: distant[distant_engine=engine](
        max_duration=60, memory=500, cpu=500, storage=500,
        edition="standard",
        system=|wrap<System>(|linux_distribution("alpine")),
        arch=_, volumes=[], containers=[], service_containers=[], tags=["example"]
    )
    startup.trigger -> request.trigger
    logRequestErrors: logErrors(label="distant")
    request.errors -> logRequestErrors.messages
}

It should be noted that the CPU and memory requirements for the Mélodium engine and possible containers are cumulative. Also, multiple different architecture cannot be requested for the same worker, so containers in the same request all have to use the same architecture. Finally, the cumuled size of all volumes must be equal or less than the Mélodium engine storage value, and each container must have storage values at least equal to the sum of the volumes mounted inside them.