Introduction

Mélodium is a language and tool to implement robust data stream, event reaction, distributed software. It proposes a wide scope of operations available for large kind of data, in very different environments, from higly constrained embedded machines to hyper scalable cloud infrastructure.

Originally designed for scientific research in audio and signal analysis, it has now been extended to handle lot of software problematics. Working on many platforms and environments, it is capable of handling large amount of data, managing auto-scaling, in real-time with high optimization.

The current book is a summary of its usage and present the purpose of Mélodium, its internal concepts, and how to program and work with it. Please refer to the Mélodium Standard Reference for detailed documentation, and to the repository for development. Also take a look to the Official Project Website.

Mélodium and this book are a work in progress, aiming to evolve and grow significantly with time. All the informations explained there might no be up-to-date compared to the current state of the project. All this work is done with passion and any comment is good to provide.

Purpose

Mélodium has been created to fill the gap between most of the programming languages and the idea of events and streams.

Most languages, while using many paradigms, see data as individual boxes that need to be moved in memory in a step-by-step approach. Developer need to define how these boxes are organized and passed as parameter, processed through loops, returned, keeping track of what's happening, and managing error cases in many locations (or assuming all will be fine else program will crash). This approach can be sufficient in many cases, and lot of languages already did a great job at it.

Now comes the problematic of events and streams.

What if we need a program able to run continously and efficiently on data without being able to assume it will fit in a box, or even in memory? What if we need it to continue to process even if somewhere or sometimes a problem happens? What if we need a program that insure everything to be ok at startup, and take care safely, by design, of errors that may occur?

Here comes Mélodium.

Installation

Mélodium can be installed through different ways. There are prebuilt binaries ready to use for major platforms, as well as installers.

Installers and binaries

For major platforms, such as Linux, Mac, and Windows, Mélodium is released as package or installer, depending on the common practice related to the operating system. Standalone binaries stored in archive files are also available for all these platforms.

Please refer to the Download Page to get installer and prebuilt binaries.

Compilation

Mélodium can be build and installed through the Rust cargo command. This method is available for all supported platforms, and require to have the Rust Environment installed and ready to work. Please refer to the Rust Project to install the Rust compiler.

To build and install Mélodium:

cargo install melodium

Please note this may take a while, depending on the network connection and machine proceeding to compilation.

Refer to the Supported Platforms chapter to check the Mélodium compatibility.

Software

In this chapter is explained the Mélodium software itself.

Its usage, the command-line interface, the possible project formats, and the supported platforms are all detailed in their own subchapters.

General Use

Mélodium, as software, can be essentially resumed as a single executable binary.

The Mélodium binary doesn't require to be at any specific location on the system, and doesn't require more permissions nor privileges than a usual executable program. This said, it have to be in the PATH of the user in order to work correctly with standalone Mélodium script and program files.

If Mélodium has been installed through a system-dedicated installation method these requirements are met, such as with an installer on Windows or package on Linux.

Mélodium software is very similar to what can be found with other programming technologies, such as Python or Java, among others. The Mélodium executable binary is the implementation of the Mélodium programming language, and receive Mélodium script and program files to execute them.

Direct Call

Mélodium can be called directly through command line, using the melodium command, followed by the name of the script or program file that has to be executed.

By default, the main entrypoint will be used, and possible following parameters will be given to the designated treatment.

A Mélodium program can declare any number of entrypoints, and having a main one is optionnal. In general, a program with a main entrypoint is aimed to be used directly, while projects without main entrypoint (either none or multiple others) are libraries.

Calling a Mélodium program is as simple as:

melodium my_program.mel

And it is strictly equivalent to:

melodium my_program.mel main

If arguments needs to be passed to the program, they can be specified as any CLI arguments:

melodium my_program.mel [ENTRYPOINT] --my_number 42 --my_boolean true

To see what entrypoints a Mélodium program proposes, use the info command:

melodium info my_program.mel

For more CLI usage details, please refer to the Command Line Interface section.

Executable Scripts and Programs

Mélodium scripts files (with .mel extension) and packaged files (with .jeu extension) can be called directly as-is if they have been designed for this use by the authors.

On POSIX systems, it require for the file to be marked as executable (x permission), and in case of .mel file, to start with the #!/usr/bin/env melodium shebang line.

This means Mélodium scripts and programs can be called as simply as any shell or general executable file:

my_program.mel --my_number 42 --my_boolean true

Command Line Interface

Mélodium is distributed as a program that have extensive and growing CLI, to see the exhaustive commands and options list:

melodium help

Run program

Launch a Mélodium program:

melodium <FILE>

or

melodium run <FILE>

or launch a specific command in Mélodium program:

melodium run <FILE> <CMD> [ARGS…]

To see specifics of a program entrypoints:

melodium run <FILE> <CMD> --help

Note: If Mélodium is installed on system, standalone .mel files and .jeu files are directly callable as-is through command line.

Program information

See the commands and options of a program:

melodium info <FILE>

Check code

Check a Mélodium program validity:

melodium check <FILE>

Generates documentation

Generates documentation, as mdBook, for a Mélodium package:

melodium doc --file <FILE> <OUTPUT>

Build Jeu files

To build a Jeu file from a project:

melodium jeu build <PROJECT> <OUTPUT_FILE>

Formats

Mélodium projects can come in three formats:

• a project tree with structure of *.mel script files;
• a standalone *.mel script file;
• a packaged project as *.jeu file.

These three formats have slightly different purposes and are detailed in their own subsection.

Project Tree

This project format consists of a tree of *.mel script files dispatched in directories, where each file and directory are their own area.

my_project/
├── baz.mel
├── Compo.toml
├── foo
│   └── bar.mel
├── foo.mel
└── main.mel

At the root of the project is a Compo.toml file, containing the information related to the project itself, such as its name, dependencies, entrypoints, and so on.

This project format is the most useful for development, as it is easily manageable with any code editor and fits with versionning systems like Git.

Standalone File

This format is, as its name suggest, a single *.mel script file.

Everything is contained in a single Mélodium script, including the information about dependencies.

A standalone project file is basically a usual *.mel file with special heading inside.

#!/usr/bin/env melodium
#! name = my_project_name
#! version = 0.1.0
#! require = std:0.8.*

/*

    Just a usual Mélodium script afterwards.

    …
*/

The details about standalone files are given in the Standalone Files chapter.

This project format is useful for quick scripts edition and deployment, prototyping, and to use as system tool.

Packaged Project

Mélodium have a project package format called Jeu. It is basically a compressed archive format with some prefixed data for Mélodium and host system handling.

When shipping a project, either for testing, deployment, or any use case better handling one-file program, the Jeu format is indicated.

To package a project as Jeu file, use the jeu subcommand:

melodium jeu build <PROJECT> <OUTPUT_FILE>

Jeu files have .jeu extension, and are directly usable as programs on systems where Mélodium is installed.

As Jeu files are already highly compressed files (using the LZMA2 algorithm), it is in general not useful to re-compress it inside other archive formats.

Supported Platforms

Mélodium supports multiple platforms. The term “Platform” refers to a set of operating system, machine architecture, and compilation method; also known as “target” by Rust developers.

Directly supported platforms are platforms for which Mélodium binaries are released.

Other platforms support

Mélodium may work on platforms that are not listed as Directly supported platforms. For those platforms, it is needed to build and install Mélodium through the Rust cargo command.

cargo install melodium

These platforms are notably Linux/BSD-like ones, as well as less common machine architecture for operating systems already supported.
For a full list of possible target, please refer:

1. to the Mélodium project repository CI checks file;
2. to the Rust platform support list, where any std-compatible target should hypothetically work.

If you have specific needs for a given platform (either getting prebuild binaries or making the compilation possible), please open a ticket on the project repository.

Specificities

Some platforms may have specificities. The aim is not to be exhaustive but to explain the reasons of these differences.

Linux GNU vs. MUSL

Without deeping dive into the details, *-gnu for Linux platforms means Mélodium rely on glibc implementation embedded by the host distribution, and *-musl means Mélodium executable is statically linked with the musl libc and so is fully autonomous in and by itself.

While both are good choice, *-gnu may not fit in some situations, such as distributions that don't ship with glibc (most notably Alpine Linux), when *-musl should work anywhere, at the cost of some extra kilobytes embedded within the executable itself.

From user perspective, no difference should be noticed in any case.

Windows GNU vs. MSVC

On Windows, *-gnu means Mélodium is built using the GNU MinGW toolchain, while *-msvc means it is built using the Microsoft Visual Studio toolchain.

Both versions are totally equivalent in terms of usage and compatibility on Windows platforms from a user and developer point of view. Difference is mainly important for low-level software developers who might want to use one over the other in some very specific situations.

Again, from user perspective, no difference should be noticed in any case. If nothing explicitly restrain the choice between both, any of them can be picked indifferently.

Programming

Section being build, please directly switch to subsections.

This chapter is oriented to developers and anyone willing to understand Mélodium logic.

Concepts

Mélodium is a programming language oriented to various data, signal and trigger treatment, process orchestration, and infrastructure management. Do to so, it relies on some concepts that are explained in this chapter.

General Orchestration

Mélodium is a language fully oriented to what happens with data. As such, in Mélodium, data and signals follows an ensemble of paths that brings it to different kind of treatments and processes.

Unlike many programming languages, Mélodium makes a total abstraction of the instruction order and don't rely on a line-by-line execution scheme. Every element can run and be shared among different threads depending on the load, data availability, or orchestration optimization decision.

To proceed with those ideas, two major elements exists in Mélodium: models and treatments. Both are essential and can summarize the whole power of Mélodium, and are briefly explained here, more extensive explanations are made in their dedicated Elements chapters.

Models

Models are elements that live through the execution of a program. They are the elements from where the events occur, data arrive, and can be shared with the outside. A filesystem agent, a HTTP server, or a SQL connection, can, among many others, be models.

Models can be declared and designed over other models, inheriting their functionnal abilities.

Models are usually instancied by some applicative treatment, and passed as parameter to their inner treatments. Most models have an immediate startup behavior, such as the JavasScriptEngine model, while some may have specific startup trigger, like the SqlPool model, that wait to have some query to do to try connection if its min_connections parameter is set to 0. Finally, some models are inherently passive, like the Environment model that defines the executive environment used for subsequent processes.

Treatments

Treatments describes flows of operations that applies on data. They can be seen as maps, on which paths connects from sources to destinations, browsing through different locations with different purposes.

Treatments can contains other treatments, as a function can call other functions. However, within a treatment the order of declaration has no importance, treatments will run when there are data ready to be processed, and all treatments can be considered as running simultaneously.

Treatments have inputs and outputs, that can be linked with connections.

Connections

Connections are a specific element, they are the ways by which the events and data are shared among treatments, and ultimately creates the functionnal logic of a program.

Connections are made the most basic element by Mélodium logic, at the core of each treatment nature. Rules are quite soft about them, and the few basic restrictions that applies are:

• a connection cannot link different kinds of data input/output;
• a connection cannot create an infinite path for data (the streaming version of infinite loops);
• multiple connections cannot connect to the same treatment input.

Inputs

Inputs are the way treatments receive data. There can be any number of input declared for a given treatment, the only constraint being that when using this treatment, all inputs it have must be connected to some output once.

Most of the treatments wait to receive some data before starting to process anything.

Outputs

Outputs are the way treatments send data. As for inputs, there can be any number of output declared for a given treatment, however any output can be connected to as many inputs as needed, or not be used at all.

Most of the treatments process data as long as their main functionnal outputs stay used, and stops when no more following treatments consume it.

Streaming

Inputs and outputs can be divided in two main categories, the streaming ones and the blocking ones. The streaming inputs and outputs are basically receiving and sending data as flow. There can be any amount of data corresponding to the associated type passed through streaming inputs and outputs. Streaming inputs and outputs are the general case of data transmission.

Blocking

The blocking inputs and outputs emit and take at most one and only one element of the given data type. This type of connection is specific for event transmission. It is generally used as trigger to start some processing.

Tracks

Tracks are the most implicit thing in Mélodium. When models are instancied and treatments connected together, it creates a potential track. The track is the whole ensemble of treatments and flows between them, that are created together, live together, and disappear together.

A track always takes its origin from a model, who request its initialization when needed and as many times as needed, for each file found, each incoming connection, or whatever the model purpose proposes. Each of them will follow the same defined chain of treatments, but on their own. This is one of the core element providing Mélodium its strong scalability.

Elements

This chapter explains the usage and implementation of Mélodium elements, that are:

• treatments,
• models,
• contexts,
• functions,
• data types.

Treatments

Treatments are the main element of the language, they can take parameters. Unlike functions, which list instructions to execute and apply changes on variables, treatments describes flows of operations that applies on data. It can be seen as a map, on which paths connects from sources to destinations, browsing through different locations with different purposes. Order of declaration has no importance, treatments will run when there are data ready to be processed, and all treatments can be considered as running simultaneously.

Within treatments are other treatments, that also take parameters, inputs, and provide outputs to the hosting treatment. Treatments are declared once, and then can be connected as many times as needed.

treatment myTreatment(var foo: u64, var bar: f64)
{
    treatmentA(alpha=foo)
    treatmentB(beta=bar, gamma=0.01)

    treatmentA.output --> treatmentB.input
}

Connections

Connections are basically paths data will follow. Connection can connect treatments outputs to inputs, but also refers to the inputs and outputs of the hosting treatment itself. A connection always links an output and an input of the same type.

use fs/util::write
use std/text/convert/string::toUtf8

treatment writeText(filename: string)
  input text: Stream<string>
  output written_bytes: Stream<u128>
{
    write(path=filename)
    toUtf8()

    Self.text -> toUtf8.text,encoded -> write.data,amount -> Self.written_bytes
}

Reference for write, toUtf8

Multiple connections from the same element are totally legal, however overloading a treatment input or a host treatment output (Self) is forbidden. Also, while omitting usage of a treatment output is legal, every input must be satisfied. Finally, all host treatment outputs must be satisfied.

Inputs and outputs (and so connections) are either streaming or blocking. A streaming connection Stream<…> is expected to send continuously values from the specified type. A blocking connection Block<…> is expected to send all-at-once. This distinction mainly rely on the core treatments that are used and the intrinsic logic applied on data. What developer should keep in mind is that streaming is the default unless blocking is required.

A specific kind of connection using the data type void exists. It is useful for transmitting information that something happens or should be triggered, schedule events, Block<void>; or to indicate continuation of something that doesn't convey data by itself, Stream<void>.

Models

Models are elements that live through the whole execution of a program. Can be declared and configured, and then instancied in a treatment declaration.

use sql::SqlPool

model MyDatabase(const max: u32 = 5) : SqlPool
{
    min_connections = 1
    max_connections = max
    url = "postgresql://my-user@my-server:4321/my_database"
}

Reference for SqlPool

Models are instancied by treatments in their prelude.

use sql::fetch
use sql::SqlPool
use std/data/block::entry
use std/data/block::get
use std/flow::trigger
use std/data::Map

treatment myApp()
  /*
    When model is instancied, it is made available within the
    treatment as if it where given as configuration parameter.
  */
  model database: MyDatabase(max=3)
  input user_id:       Block<string>
  output user_name:    Block<Option<string>>
  output user_level:   Block<Option<u32>>
  output user_failure: Block<string>
{
    userData[database=database]()

    Self.user_id -> userData.user_id,user_name -> Self.user_name
                    userData.user_level --------> Self.user_level
                    userData.failure -----------> Self.user_failure
}

/*
  Model can be given as configuration parameter.
*/
treatment userData[database: SqlPool]()
  input user_id:     Block<string>
  output user_name:  Block<Option<string>>
  output user_level: Block<Option<u32>>
  output failure:    Block<string>
{

    /*
      And then passed again as configuration parameter.
    */
    fetchUser: fetch[pool=database](
      sql = "SELECT name, level IN users WHERE id = ? LIMIT 1",
      bindings = ["user_id"]
    )
    entry<string>(key="user_id")
    trigger<Map>()

    Self.user_id -> entry.value,map -> fetchUser.bind,data -> trigger.stream

    getUserName: get<string>(key="name")
    getUserLevel: get<u32>(key="level")

    trigger.first --> getUserName.map,value -> Self.user_name
    trigger.first -> getUserLevel.map,value -> Self.user_level

    fetchUser.failure -> Self.failure
}

Reference for fetch, entry, get, trigger

myApp

userData

In most cases, models are instancied internally by treatments and not exposed, user developer can make direct call on model-dependent treatments without instancing its own, just giving required parameters to the sequence. The cases where user may give its own defined model is to configure elements such as external software connections or interfaces.

Contexts

Contexts are data available through a whole track. Unlike parameters, they inherently exists and are accessible by all treatements requiring them, as long as the source of the track they are in provide it.

The calling treatments don't have to explicitly pass any context to their inner called treatements. Contexts have special naming convention, starting with @.

Context providing and requirement

Sources provide contexts, and treatments connected afterwards this source can require context to access it.

Requiring a context means the treatment can only be used in a track coming from a source providing such context.

use http/server::HttpServer
use http/server::connection
use http/method::|get as |methodGet
use http/server::@HttpRequest
use std/data::|get

treatment myApp[http_server: HttpServer]() {
    connection(method=|methodGet(), route="/user/:user")
    actionGetUser()

    connection.data -> actionGetUser.data,result -> connection.data
}

treatment actionGetUser()
  require @HttpRequest
  input  data:   Stream<byte>
  output result: Stream<byte>
{
    getUser(user_id = |get<string>(@HttpRequest[parameters], "user"))

    Self.data -> getUser.data,result -> Self.result
}

treatment getUser(user_id: Option<string>)
  input data   :  Stream<byte>
  output result: Stream<byte>
{
    // Do some stuff…
}

Reference for HttpServer, connection, |get (http method), @HttpRequest,|get (map access)

myApp

actionGetUser

As contexts comes at track creation, they are inherently variable data.

Functions

Functions are directly callable elements, as in any other programming language, they take parameters and return a value.

Functions in Mélodium are always pure, meaning they don't have any side effects.

Functions are executed at program initialisation or track creation, depending if their return value is used as constant or variable parameter.

Functions are recognizable by the | symbol starting their name.

Data types

Data types are very basically the definition of data that can be given as parameter, transmitted through inputs and outputs, and more generally used across functions.

Mélodium packages can define their own data types, completing the core types, and they can be use-d in any place data type can be given.

One of the most common data type is the standard Map.

Data types can implements traits and be used as generics in some places.

Core types

Mélodium have multiple core data types, shared across four main categories:

• unsigned integers,
• signed integers,
• floating-point numbers,
• textual data,

on which bool, byte and void can be added.

All those types are described across their respective section, each type meets a specific purpose.

Byte

A byte is basically the most atomic unit of data manipulable through Mélodium. It represents any 8-bits data, without more assumption on what it could be.

Bool

A bool is a boolean value that can be either set to true or false. Conversion treatments are available for bools to be turned into bytes, numbers, or any kind of value.

Void

void data type does not hold any value, it just indicates that something is existing. It is used through connections to transmit triggers or streaming indicators.

Unsigned integers

Signed integers

Floating-point numbers

Floating-point numbers are defined in IEEE 754-2008. They can mostly be considered as decimal numbers, for a deeper explanation, please refers to the Single-precision floating-point format (for f32) and Double-precision floating-point format (for f64) articles on Wikipedia.

They can store positive or negative values, but also be in one of those three states:

• positive infinity, can be result of something like 1.0/0.0;
• negative infinity, can be result of something like -1.0/0.0;
• not a number, can be result of a square root of negative number (aka. complex number).

Textual data

All textual information is represented as Unicode. A char uses 4 bytes to store any Unicode scalar value, as defined in Unicode Standard. Unlike many other programming languages, Mélodium does not assume a char and a byte (nor combination of bytes) to be equivalent at all, for many reasons such as:

• a byte only have 256 values, while all human languages combined have much more "letters";
• a letter in Unicode Text Format can be up to 4 bytes;
• lot of values are illegal according to Unicode;
• Unicode standard provide a strong universality of what textual data can be represented;
• making data types reliable, each one having its own purpose, then char guarantees valid text data while byte only assume it is data.

The string data type can represent any UTF-8 text and its size depends on the length of the text. Interestingly, strings are not a combination of chars, but real UTF-8 strings. Taking the text Mélodium and putting it as vector of chars, 32 bytes (8 chars × 4 bytes) are used, but as string only 9 bytes. This technical subtility is transparent for users and conversion treatments are provided if needed.

Mélodium can handle many encodings through its encoders and decoders, taking and providing byte streams.

Parameters

Treatments and models declares parameters. Parameters are like in any language: elements given by the caller to set up behavior of the model or treatment.

Const and var

In Mélodium, parameters can be either constant or variable, respectively declared with keywords const and var. A constant parameter designates something that will keep the same value during all the execution, on all tracks generated through the given call. They are used mostly to configure models, that have all parameters required to be constant. A variable parameter designates something that may have different values on each track generated.

While a constant can be used to set up constant and variable parameters, variable elements (parameters but also contextes) can only be used to set up other variables.

Generics

Generics are a mechanism allowing to rely on type abstraction to process data.

use std/ops/num::isPositive
use std/conv::saturatingToI64

treatment demonstration()
  input  floating_point_value: Stream<f32>
  output integer_value: Stream<i64>
  output is_positive: Stream<bool>
{
    // Treatments isPositive and saturatingToI64 are depending on generic type,
    // that have to be given at instanciation.
    isPositive<f32>()
    saturatingToI64<f32>()

    Self.floating_point_value ------> isPositive.value,positive --> Self.is_positive
    Self.floating_point_value -> saturatingToI64.value,into ------> Self.integer_value
}

Reference for isPositive, saturatingToI64

Traits restriction

Instead of allowing any kind of data, generics can be restrained to specific traits, requiring the given data type to implement these traits to be used with the element.

use std/ops/num::isPositive
use std/conv::saturatingToI64

// A treatment can have generic parameter, with optionnal trait restrictions,
// in order to fit its functionnal abilities.
treatment demonstration<N: Float + SaturatingToI64>()
  // Generics can be used at any place a type can be given.
  input  floating_point_value: Stream<N>
  output integer_value: Stream<i64>
  output is_positive: Stream<bool>
{
    // As N fits the Float and SaturatingToI64 traits,
    // it can be passed to those treatments too.
    isPositive<N>()
    saturatingToI64<N>()

    Self.floating_point_value ------> isPositive.value,positive --> Self.is_positive
    Self.floating_point_value -> saturatingToI64.value,into ------> Self.integer_value
}

Traits

Traits are intrinsic abilities of a given type.

Any type can implement any trait, as long as it have a logical meaning.

Each type comes with its own list of implemented traits, and the respective type documentation should be reffered to know these ones.

The full list of existing traits in Mélodium is presented in next section, as well as the core types traits implementations.

Traits list

Here is the exhaustive list of Mélodium traits.

Some traits are grouped by family, as they behave the same way. The behavior is then explained with general meaning, and each individual trait description exposes some specificities.

Infaillible conversions

Infaillible conversions designates conversions that are guaranteed to succeed, for which for every X initial state there exist a Y result state.

While these conversions are guaranteed to succeed, they are not guaranteed to be reversible. As example, u8 implements ToI64, but i64 does not implements ToU8.

These traits are mostly useful trough treatments and functions of the conv area.

ToI8

Type is able to convert itself into a i8 value.

ToI16

Type is able to convert itself into a i16 value.

ToI32

Type is able to convert itself into a i32 value.

ToI64

Type is able to convert itself into a i64 value.

ToI128

Type is able to convert itself into a i128 value.

ToU8

Type is able to convert itself into a u8 value.

ToU16

Type is able to convert itself into a u16 value.

ToU32

Type is able to convert itself into a u32 value.

ToU64

Type is able to convert itself into a u64 value.

ToU128

Type is able to convert itself into a u128 value.

ToF32

Type is able to convert itself into a f32 value.

ToF64

Type is able to convert itself into a f64 value.

ToBool

Type is able to convert itself into a bool value.

ToByte

Type is able to convert itself into a byte value.

ToChar

Type is able to convert itself into a char value.

ToString

Type is able to convert itself into a string value.

While it is useful to have a conversion to string for a type, it should not be considered as a “readable” version of the value, and so not be confused with Display trait, that is especially designed for this purpose.

Faillible conversions

Faillible conversions designates conversions that are possible without being guaranteed to succeed.

Those conversions are useful to get an Option<T> result, where T is the target type, containing a value if conversion succeed, or none if it cannot be done.

These traits are mostly useful trough treatments and functions of the conv area.

A general rule is that when a type implements a To* trait, it also does for its TryTo* counterpart.

TryToI8

Type can try to convert itself into a i8 value.

TryToI16

Type can try to convert itself into a i16 value.

TryToI32

Type can try to convert itself into a i32 value.

TryToI64

Type can try to convert itself into a i64 value.

TryToI128

Type can try to convert itself into a i128 value.

TryToU8

Type can try to convert itself into a u8 value.

TryToU16

Type can try to convert itself into a u16 value.

TryToU32

Type can try to convert itself into a u32 value.

TryToU64

Type can try to convert itself into a u64 value.

TryToU128

Type can try to convert itself into a u128 value.

TryToF32

Type can try to convert itself into a f32 value.

TryToF64

Type can try to convert itself into a f64 value.

TryToBool

Type can try to convert itself into a bool value.

TryToByte

Type can try to convert itself into a byte value.

TryToChar

Type can try to convert itself into a char value.

TryToString

Type can try to convert itself into a string value.

Saturating conversions

Saturating conversions are specific kind of conversions where a type can try to convert itself into another one, and instead of failing the conversion if its value cannot be rendered into the target type, push to the closest bound of that type according to initial value.

This kind of trait exists to target numeric types.

SaturatingToI8

Type can make a saturating conversion to i8 value.

SaturatingToI16

Type can make a saturating conversion to i16 value.

SaturatingToI32

Type can make a saturating conversion to i32 value.

SaturatingToI64

Type can make a saturating conversion to i64 value.

SaturatingToI128

Type can make a saturating conversion to i128 value.

SaturatingToU8

Type can make a saturating conversion to u8 value.

SaturatingToU16

Type can make a saturating conversion to u16 value.

SaturatingToU32

Type can make a saturating conversion to u32 value.

SaturatingToU64

Type can make a saturating conversion to u64 value.

SaturatingToU128

Type can make a saturating conversion to u128 value.

SaturatingToF32

Type can make a saturating conversion to f32 value.

SaturatingToF64

Type can make a saturating conversion to f64 value.

Bounded

The type have minimum and maximum limits.

This trait is mostly useful trough functions of the types area.

Binary

Type can be used for meaningful mathematical binary operations.

This trait is mostly useful trough treatments and functions of the bin area.

Signed

Type is signed, meaning it can have negative values.

This trait is mostly useful trough treatments and functions of the num area.

Float

Type represent floating-point values, and is able to proceed to floating-point arithmetic.

This trait is mostly useful trough treatments and functions of the float area.

PartialEquality

Type can be compared to itself and tell wether both values are equal or not.

PartialEquality does not require a full equivalence between all the value of a type. As example, for f32 and f64, NaN is not equal to itself. As such, those types implements PartialEquality but not Equality.

This trait is mostly useful trough treatments and functions of the ops area.

Equality

Type can be compared to itself and tell wether both values are equal or not.

Equality require a full equivalence between all the value of a type, meaning any X value is always equal to itself and always different from any other.

This trait is mostly useful trough treatments and functions of the ops area.

PartialOrder

Type implements partial order, meaning values of this type can be compared in a way telling if one is greater or lesser to another, strictly or not.

This trait is mostly useful trough treatments and functions of the ops area.

Order

Type implements total order, meaning values of this type can be compared, and absolute minimums and maximums in a set of those values can be found.

This trait is mostly useful trough treatments and functions of the ops area.

Basic arithmetic

Those traits corresponds to basic arithmetic operations, that are guaranteed to give a result, while that one may not be meaningful in some cases.

Those operations can most notably be subject to overflow, meaning the result value of an operation cannot fit into the type on which it is applied.

This trait is mostly useful trough treatments and functions of the num area.

Add

Type can proceed to addition between two values.

Sub

Type can proceed to substraction of two values.

Mul

Type can proceed to multiplication of two values.

Div

Type can proceed to division of a value by another.

Rem

Type can proceed to division of a value by another and give the remainder.

Neg

Type can negates its values.

Pow

Type can elevates its values by an exponent.

Euclid

Type can proceed to euclidian division of a value by another.

Checked arithmetic

Those traits corresponds to basic arithmetic operations, that may have meaningless result.

Those operations can avoid overflows, as the result value of an operation that cannot fit into the type is ignored.

This trait is mostly useful trough treatments and functions of the num area.

CheckedAdd

Type can proceed to addition between two values and avoid meaningless results.

CheckedSub

Type can proceed to substraction of two values and avoid meaningless results.

CheckedMul

Type can proceed to multiplication of two values and avoid meaningless results.

CheckedDiv

Type can proceed to division of a value by another and avoid meaningless results.

CheckedRem

Type can proceed to division of a value by another and give the remainder, while avoiding meaningless results.

CheckedNeg

Type can negates its values and avoid meaningless results.

CheckedPow

Type can elevates its values by an exponent and avoid meaningless results.

CheckedEuclid

Type can proceed to euclidian division of a value by another and avoid meaningless results.

Saturating arithmetic

Those traits corresponds to basic arithmetic operations, that saturates the value to the closest bound to truth if the result cannot fit into the type.

This trait is mostly useful trough treatments and functions of the num area.

SaturatingAdd

Type can proceed to addition between two values, and bound to minimum or maximum value if addition result is out of range.

SaturatingSub

Type can proceed to substraction of two values, and bound to minimum or maximum value if substraction result is out of range.

SaturatingMul

Type can proceed to multiplication of two values, and bound to minimum or maximum value if multiplication result is out of range.

Wrapping arithmetic

Those traits corresponds to basic arithmetic operations, that wraps on purpose when the operation result goes out of range for the subject type.

This trait is mostly useful trough treatments and functions of the num area.

WrappingAdd

Type can proceed to addition between two values, and wrap over its value range if addition exceeds type boundaries.

WrappingSub

Type can proceed to substraction of two values, and wrap over its value range if substraction exceeds type boundaries.

WrappingMul

Type can proceed to multiplication of two values, and wrap over its value range if multiplication exceeds type boundaries.

WrappingNeg

Type can negates its values, and wrap over its value range if negation exceeds type boundaries.

Hash

Type is subject to hash, and so can be used as key to reference data.

Serialize

Type implements serialization, meaning it can be turned into linear data and send out from a program.

Deserialize

Type implements deserialization, meaning it can be received from outside of a program and parsed to build a value.

Display

Type can be rendered as human readable string and possibly displayed to users in a way making sense for them.

Display trait is different to the ToString trait as it does not have the same functionnal purpose. ToString is expected to be a technical conversion of data, while Display is meant to expose it to human eyes.

Script files

Launch

Mélodium scripts can be launched using melodium command:

melodium main_script.mel

Or if the main script includes a shebang:

./main_script.mel

Recommended shebang being #!/usr/bin/env melodium.

Script file must contains basic identity informations to be used as entrypoint:

• name (required),
• version (optional, in semver sematic),
• requirements (optional).

#!/usr/bin/env melodium
#! name = my_script
#! version = 0.1.0
#! require = std:0.8.0 fs:0.8.0 …

// Content

Note about encoding

Mélodium script files are plain UTF-8 text, without byte order mark. This choice is made for three main reasons:

1. a choice on encoding, even arbitrary, is better than no choice;
2. Unicode provides the wider support for any characters from all human languages and scripts, existing and future, ensuring continuity;
3. the Mélodium engine is implemented in Rust, itself natively representing text as UTF-8.

Project Organization

Mélodium projects are organized through a root folder, containing a Compo.toml file, and .mel files.

my_project/
├── Compo.toml
├── baz.mel
├── foo
│   └── bar.mel
├── foo.mel
└── main.mel

Areas

In Mélodium, an area is an element location. It is a concept similar to Rust modules or Java locations.

An area is materialized by a .mel file, that contains all its elements. Subareas are made using a folder having the area name, and creating .mel files inside.

my_project/
├── Compo.toml
├── baz.mel
├── foo     # folder for subareas
│   └── bar.mel
├── foo.mel # area
└── main.mel

In this example, the foo area is developed within the foo.mel file, and bar area is located within the foo/ folder to be a subarea of foo. They can respectively be called through:

• root/foo::<element>
• root/foo/bar::<element>

Current project references

In Mélodium, project code can refer to its own content using the root and local keywords, and do not use the name of the project itself.

In the example, to refer an element that is present in the baz area, the call root/baz::<element> must be used. Similarly, if foo area refers to something within bar, local/bar::<element> could be written.

Dependencies

Mélodium is not only a language and execution engine, but a dependency manager too.

Mélodium, as dependency manager, is largely inspired from Rust Cargo. If you're already familiar with the way cargo work with dependencies, you shouldn't discover new things here.

The dependency and package management in Mélodium is still at its early stage, and things may change about it in the future. Information presented here is about actual state and not written in stone.

Dependency list

Every Mélodium project have a dependency list, that it requires to work. This list is made in the Compo.toml file.

A usual dependency list can be:

• std >=0.8.0
• http >=0.8.0
• fs >=0.8.0

Note that the std dependency in Mélodium have to be explicitly given. This is mainly because standard library is in quick evolution and require to rely on precise version.

Dependency tree

Each project having its own dependency list, using a project means building a dependency tree. Mélodium is quite cool about dependencies management, but have some rules:

• a project cannot rely directly on other versions of that same project (but multiple versions can appear in the dependency tree);
• circular dependencies are forbidden, meaning a project of a given version cannot appear anywhere in its own dependency tree.

Semantic Versionning

Mélodium follows the semantic versionning norm SemVer. This versionning system aims to be consistent with software evolution and avoid custom version designation caveats.

A version is always designated as x.y.z, with optionnal pre-release identifier added at the end as x.y.z-rc1.

This page is essentially an adaptation from the Cargo Rust manual to Mélodium, from which the versionning and dependency system is largely inspired.

Version designation

[dependencies]
regex = "0.8.1"

The string "0.8.1" is a version requirement. Although it looks like a specific version of the regex package, it actually specifies a range of versions and allows SemVer compatible updates. An update is allowed if the new version number does not modify the left-most non-zero number in the major, minor, patch grouping. In this case, we may have version 0.8.3 if it is the latest 0.8.z release, but would not update us to 0.9.0. If instead we had specified the version string as 1.0, cargo should update to 1.1 if it is the latest 1.y release, but not 2.0. The version 0.0.x is not considered compatible with any other version.

Here are some more examples of version requirements and the versions that would be allowed with them:

1.2.3  :=  >=1.2.3, <2.0.0
1.2    :=  >=1.2.0, <2.0.0
1      :=  >=1.0.0, <2.0.0
0.2.3  :=  >=0.2.3, <0.3.0
0.2    :=  >=0.2.0, <0.3.0
0.0.3  :=  >=0.0.3, <0.0.4
0.0    :=  >=0.0.0, <0.1.0
0      :=  >=0.0.0, <1.0.0

This compatibility convention is different from SemVer in the way it treats versions before 1.0.0. While SemVer says there is no compatibility before 1.0.0, Mélodium considers 0.x.y to be compatible with 0.x.z, where y ≥ z and x > 0.

It is possible to further tweak the logic for selecting compatible versions using special operators as described in the next section.

Use the default version requirement strategy, e.g. std = "1.2.3" where possible to maximize compatibility.

Version requirement syntax

Caret requirements

Caret requirements are the default version requirement strategy. This version strategy allows SemVer compatible updates. They are specified as version requirements with a leading caret (^).

^1.2.3 is an example of a caret requirement.

Leaving off the caret is a simplified equivalent syntax to using caret requirements. While caret requirements are the default, it is recommended to use the simplified syntax when possible.

log = "^1.2.3" is exactly equivalent to log = "1.2.3".

Tilde requirements

Tilde requirements specify a minimal version with some ability to update. If you specify a major, minor, and patch version or only a major and minor version, only patch-level changes are allowed. If you only specify a major version, then minor- and patch-level changes are allowed.

~1.2.3 is an example of a tilde requirement.

~1.2.3  := >=1.2.3, <1.3.0
~1.2    := >=1.2.0, <1.3.0
~1      := >=1.0.0, <2.0.0

Wildcard requirements

Wildcard requirements allow for any version where the wildcard is positioned.

*, 1.* and 1.2.* are examples of wildcard requirements.

1.*   := >=1.0.0, <2.0.0
1.2.* := >=1.2.0, <1.3.0

Note: Mélodium does not allow bare * versions.

Comparison requirements

Comparison requirements allow manually specifying a version range or an exact version to depend on.

Here are some examples of comparison requirements:

>= 1.2.0
> 1
< 2
= 1.2.3

Multiple version requirements

As shown in the examples above, multiple version requirements can be separated with a comma, e.g., >= 1.2, < 1.5.

Entrypoints

Mélodium projects can have entrypoints. As their name suggest, they are the treatments that can be called directly to start a Mélodium program.

A project is not required to have entrypoints, if it is a library as example. On the opposite, a project can have multiple entrypoints, if that project is a program that can be used in different situations.

Naming entrypoints

Entrypoints are essentially a name associated with a treatment path, like:

• server: my_project/foo::serve
• client: my_project/bar::request
• main: my_project/etc::main

Entrypoints names have same restriction as treatments names but don't have to be the same as the treatment they designates.

The entrypoint name is expected to be typed in command line, as commands of the program:

// Starts 'my_program' with 'server' entrypoint
$ my_program.jeu server

// With explicit melodium command
$ melodium my_program.jeu server

An exception is made for the main entrypoint, that if present, is called directly if no specific entrypoint is given to program.

// Starts 'my_program' with 'main' entrypoint
$ my_program.jeu

// With explicit melodium command
$ melodium my_program.jeu

Entrypoints parameters

If a treatment used as entrypoint have parameters, they automatically becomes acceptable command arguments.

// In root/foo
treatment serve(bind: string = "localhost", port: u16) 
{
    /* 
        Implementation…
    */
}

With entrypoint server: my_project/foo::serve.

$ my_program.jeu server --port 6789 --bind '"192.168.55.66"'

Note: in actual development state, arguments given through CLI must match Mélodium syntax. As such, string parameters must be quoted with ". This is a trade-off to allow complex structures, like arrays, to be passed as arguments. This may be changed in future.

Compo.toml

Compo.toml file is the very file making a folder being a Mélodium project.

Its structure is really basic, containing few informations about the project:

name = "my_project"
version = "0.0.1"

[dependencies]
encoding = "0.8.0"
fs = "0.8.0"
http = "0.8.0"
javascript = "0.8.0"

[entrypoints]
main = "my_project/main::main"

Name and version

The name field contains the very name of the project, that will be exposed in repository and used by dependents projects to call it.

Dependencies

The dependencies list contains the names of dependencies and version requirements, as explained in the dedicated chapter.

Entrypoints

The entrypoints list contains the named entrypoints and targeted treatments.

Standalone Files

Mélodium is able to handle standalone .mel files. Opposite to .jeu files that are packaged Mélodium applications, .mel standalone files aims to stay as simple scripts, for use cases as quick deployment, small tooling, or administration helpers.

Standalone Mélodium files are really usual .mel files with a special heading.

#!/usr/bin/env melodium
#! name = my_project_name
#! version = 0.1.0
#! require = std:0.8.*

/*

    Just a usual Mélodium script afterwards.

    …
*/

They always start with the #!/usr/bin/env melodium shebang, allowing them to be used as system script. This shebang is mandatory as it is also used by Mélodium engine to ensure script was designed to be called as-is.

name and version fields works the same as in Compo.toml. require is expected to be the list of dependencies, separated by spaces, in with the <name>:<version_requirement> format. require can also be repeated multiple times if needed.

Unlike other Mélodium projects, standalone Mélodium files are required to have one and only one entrypoint, that is always set up to the main treatment. All the characteristics relevant to treatement parameters and CLI arguments are applicable as for any entrypoint.

Reference

The Mélodium reference is available on doc.melodium.tech. The whole Standard Library is documented there, and can be browsed through areas.

Runtime

Mélodium uses a runtime engine. The script files are fully parsed and their logic build and checked before any execution starts. When launching a Mélodium script, multiple stages happens:

1. Script textual parsing and semantic build
2. Usage and depedencies resolution
3. Logic building
4. Models instanciation
5. Execution and tracks triggering

Release

Releasing a Mélodium project as application aims to be extremely simple. It mostly consists in packaging it as Jeu file and distribute it.

Package

To build a packaged Mélodium application, run the melodium jeu build command:

$ melodium jeu build <PROJECT_LOCATION> <OUTPUT_FILE>

This will check the given project and package it in a resulting .jeu file.

Run

To run a packaged Mélodium application, the host system must have Mélodium installed.

Considering the file is myapp.jeu, it can be run through direct call or with explicit melodium command:

// Direct call
$ ./myapp.jeu

// Explicit Mélodium call
$ melodium run myapp.jeu

Distribute

Jeu files can be distributed as-is. The host machine only need Mélodium to be installed.

Jeu files are already highly compressed data (using the LZMA2 algorithm), storing and distributing them in re-compressed files is not useful.

Containers

Mélodium applications can be easily containerized by putting them in a container image alongside a Mélodium installation.

This chapter is being build alongside the Mélodium container images design.

Information to come soon.

About the author

Quentin Vignaud is IT engineer graduated from CESI, and M.Sc. in computing science from UQÀM. Working at Doctolib as data and software engineer, originally authored Mélodium during studies at UQÀM, while doing scientific research in music analysis.

Website: https://www.quentinvignaud.com/