Secrets

Passwords, tokens and keys are given to programs as Secret<T> values, most often Secret<string>. A secret holds where its value comes from rather than the value itself, and only the elements that need the value can reveal it, at the moment they use it. Its value never appears in parameters, logs, debug files or reports, and never in command arguments.

Locating secrets

A secret parameter takes a locator, <scheme>:<path>, telling where the value is read from when revealed:

Programs only get the environment variables whose name starts with MELODIUM_SECRET_: the other ones, such as those configuring Mélodium itself, stay out of their reach. Whoever runs a program exposes a variable to it on purpose, for example MELODIUM_SECRET_GITHUB_TOKEN: ${{ github.token }} in a GitHub workflow.

The env: and file: sources are part of Mélodium builds that have access to an environment and a file system, which is not the case of the WebAssembly one.

Locators are written as string literals, for defaults as well as for arguments:

use sql::SqlPool
use std/ops/option::|wrap

treatment database(const password: Secret<string> = "env:MELODIUM_SECRET_DB_PASSWORD")
  model pool: SqlPool(url="postgresql://app@db.local/app", password=|wrap<Secret<string>>(password))
{
}

On the command line, the same parameter takes a locator, never a value:

melodium run app.mel --password file:/run/secrets/db_password

--check-secrets resolves the secrets given as parameters before running, so that a missing variable or file fails right away rather than when first used.

The functions of std/secret also build secrets: |from_environment and |from_file read from the environment and from files, and |locate takes a locator and a policy (see below). A value only known at runtime, such as a token given by an API response, becomes a secret through the conceal treatment.

Using secrets

Elements receiving secrets reveal them only when they use them:

• sql::SqlPool puts its password into the connection URL when connecting, and refuses query bindings holding secrets.
• http/client::HttpClient adds its secret_headers to each request.
• process environments give secret_variables and a secret standard input to commands, never through their arguments. In Kubernetes pods, commands receiving secret variables run through /bin/sh, which their image needs.
• work/distant::DistantEngine and container pull secrets are revealed to send the worker request.
• work/access::|new_access takes the authentication keys of a distribution engine as secrets, revealed by distrib::start to authenticate the connection.

Incoming HTTP requests give their Authorization and Cookie headers as the authorization and cookie secrets of the @HttpRequest context, and leave them out of the request headers. They come from the client, so std/secret::reveal may reveal them, for example to compare a token. A server can list other headers to give only as secrets, such as the token of GitLab webhooks: HttpServer(..., secret_headers="X-Gitlab-Token") puts them in the secret_headers map of @HttpRequest. The crypto package checks such values without revealing them to the program: crypto/token::check compares a received token with a secret in constant time, and crypto/signature/hmac::verify checks the HMAC signature of a body, such as the X-Hub-Signature-256 header of GitHub webhooks, with a key given as a secret. crypto/signature/ed25519::verify checks Ed25519 signatures, such as those of Discord interactions, and crypto/token/jwt::verify checks received JSON Web Tokens against a shared secret or a public key, such as the OIDC tokens of CI jobs.

The crypto package also makes secrets: crypto/token/jwt::sign signs a JSON Web Token with a secret key and gives it as a secret, such as to authenticate as a GitHub App, and crypto/random::token generates random tokens.

Secrets can be built from other secrets without being revealed. |format fills a template with secrets and strings, |base64 and |url_encode encode them, and the result is computed only when revealed:

use http/client::HttpClient
use std/data/map::Map
use std/data/map::|map as |secret_map
use std/data/map::|entry as |secret_entry
use std/data/string_map::|map
use std/ops/option::|wrap
use std/secret::|format

treatment api(const token: Secret<string> = "env:MELODIUM_SECRET_API_TOKEN")
  model client: HttpClient(
    base_url=|wrap<string>("https://api.example.com"),
    headers=|map([]),
    secret_headers=|wrap<Map>(|secret_map([
        |secret_entry<Secret<string>>("Authorization", |format(
            "Bearer {token}",
            |secret_map([|secret_entry<Secret<string>>("token", token)]),
            "authorization"
        ))
    ]))
  )
{
}

Commands get secrets as environment variables or on their standard input:

use process/command::|command
use process/environment::Environment
use process/environment::|environment
use process/environment::|with_secret_variables
use process/local::execOnce
use std/data/map::|map as |secret_map
use std/data/map::|entry as |secret_entry
use std/data/string_map::|map
use std/engine/util::startup
use std/ops/option::|wrap

treatment deploy(const token: Secret<string> = "env:MELODIUM_SECRET_DEPLOY_TOKEN")
{
    startup()
    deploy: execOnce(
        command=|command("deploy", ["--target", "production"]),
        environment=|wrap<Environment>(|with_secret_variables(
            |environment(|map([]), _, false, false),
            |secret_map([|secret_entry<Secret<string>>("DEPLOY_TOKEN", token)])
        ))
    )
    startup.trigger -> deploy.launch
}

Some secrets are only known at runtime, such as a token concealed by std/secret::conceal from an API response, or a secret read by vault::get from a path built at runtime. They arrive on inputs: the treatments of std/secret/block derive and narrow them as the functions of std/secret do for parameters, process/environment/block adds them to command environments, and http/client::requestWithSecretHeaders sends them as headers. None of them reveals the secret before the request or the command needs it:

use http/client::HttpClient
use http/client::requestWithSecretHeaders
use http/method::|get
use std/data/map/block::entry
use std/data/string_map::StringMap
use std/secret/block::format
use vault::Vault
use vault::get

treatment report[vault: Vault, client: HttpClient]()
  input  token_path: Block<string>
  input  url:        Block<string>
  input  headers:    Block<StringMap>
  output data:       Stream<byte>
{
    token: get[vault=vault](name="tenant_token")
    tokenEntry: entry<Secret<string>>(key="token")
    bearer: format(template="Bearer {token}", name="authorization")
    headerEntry: entry<Secret<string>>(key="Authorization")
    request: requestWithSecretHeaders[client=client](method=|get())

    Self.token_path -> token.path,secret -> tokenEntry.value,map -> bearer.entries,secret -> headerEntry.value,map -> request.secret_headers
    Self.url -> request.url
    Self.headers -> request.req_headers
    request.data -> Self.data
}

vault::get gives secrets the default policy unless given transmission and plain_reveal, as |locate takes them.

std/secret::reveal gives the value of a secret as ordinary data. It is refused unless the policy of the secret allows it, as the value then flows like any other data. Secrets from the environment and from files are never plainly revealed: a program only gets them as secrets, revealed by the elements using them.

Policies

Each secret has a policy, set when it is created, and that can only be narrowed afterwards:

• transmission, where the secret may go when the program is distributed: local (never leaves the engine, the default), reference (only the locator is sent, and the distant engine resolves it with its own sources) or value (the value is sent, over TLS only);
• reveal, which elements may reveal it: any of them by default, or only those listed;
• plain reveal, whether std/secret::reveal may reveal it: not by default, and never for secrets from env: and file:.

Locator literals, |from_environment and |from_file give the default policy. |locate and conceal take the transmission and plain reveal explicitly, and |keep_local, |limit_to_reference, |reveal_only_by and |forbid_plain_reveal narrow a policy:

use std/secret::|from_environment
use std/secret::|locate
use std/secret::|reveal_only_by

// Sent by value to distant engines, never plainly revealed.
|locate<string>("vault:kv/data/ci#token", "ci_token", "value", false)

// Only revealed by `SqlPool`.
|reveal_only_by<string>(|from_environment("MELODIUM_SECRET_DB_PASSWORD", "db_password"), ["sql::SqlPool"])

A secret built from others gets the most restrictive combination of their policies.

Audit

Every reveal, and every secret sent to a distant engine, is recorded as a debug event naming the element and its label. Refused and failed reveals are also written to the log, and are errors of the revealing element.

Values revealed during a run are replaced by <secret "name"> in log messages, including their lines, base64 and URL-encoded forms, as long as they are at least 8 bytes long. Data streams are left unchanged: a value made plain by reveal appears in detailed debug events like any other data.

Plaintext held by Mélodium is overwritten once not needed anymore, and before the process exits, including on interruption. This is best effort: memory written to swap or core dumps, and copies made by libraries or programs given the value, are out of its reach.

Distribution

When a treatment runs on a distant engine, its parameters and the data sent to it follow the transmission of each secret they hold, including secrets inside data values such as process environments:

• local secrets are refused, with an explicit error;
• reference secrets are resolved by the distant engine, with its own environment, files and sources;
• value secrets are resolved by the sending engine and sent over TLS, never over plain TCP.

The distant engine keeps the policy of the secrets it receives, and enforces it.