YAML

The NOS-T Tools library requires a Yet Another Markup Language (YAML) configuration file. This file adopts a format similar to AsyncAPI to define the connection configuration for the RabbitMQ broker and Keycloak authentication servers.

Configuration Overview

The YAML files is composed of the following sections:

  1. Info: Contains metadata about the YAML configuration file.

  2. Servers: Contains the connection configuration for the RabbitMQ broker and Keycloak authentication servers.

  3. Execution: Contains the execution configuration for the NOS-T Tools library.

  4. Channels: Contains the channel configuration for the NOS-T Tools library.

Each section is explained in detail below.

Info Section

Contains metadata about the configuration file. This does not influence the operation of the NOS-T Tools library, it is merely for documentation and informational purposes.

Example:

info:
  title: Novel Observing Strategies Testbed (NOS-T) YAML Configuration
  version: '1.0.0'
  description: Version-controlled configuration file for Snow Observing Systems (SOS) project

Servers Section

Defines connection parameters for the RabbitMQ message broker and Keycloak authentication server.

RabbitMQ Configuration

pydantic model RabbitMQConfig

Show JSON schema
{
   "title": "RabbitMQConfig",
   "type": "object",
   "properties": {
      "keycloak_authentication": {
         "default": false,
         "description": "Keycloak authentication for RabbitMQ.",
         "title": "Keycloak Authentication",
         "type": "boolean"
      },
      "host": {
         "default": "localhost",
         "description": "RabbitMQ host.",
         "title": "Host",
         "type": "string"
      },
      "port": {
         "default": 5672,
         "description": "RabbitMQ port.",
         "title": "Port",
         "type": "integer"
      },
      "tls": {
         "default": false,
         "description": "RabbitMQ TLS/SSL.",
         "title": "Tls",
         "type": "boolean"
      },
      "virtual_host": {
         "default": "/",
         "description": "RabbitMQ virtual host.",
         "title": "Virtual Host",
         "type": "string"
      },
      "message_expiration": {
         "default": "60000",
         "description": "RabbitMQ expiration, in milliseconds.",
         "title": "Message Expiration",
         "type": "string"
      },
      "delivery_mode": {
         "default": 2,
         "description": "RabbitMQ delivery mode (1: non-persistent, 2: durable).",
         "title": "Delivery Mode",
         "type": "integer"
      },
      "content_type": {
         "default": "text/plain",
         "description": "RabbitMQ MIME content type (application/json, text/plain, etc.).",
         "title": "Content Type",
         "type": "string"
      },
      "heartbeat": {
         "default": 30,
         "description": "RabbitMQ heartbeat interval, in seconds.",
         "title": "Heartbeat",
         "type": "integer"
      },
      "connection_attempts": {
         "default": 3,
         "description": "RabbitMQ connection attempts before giving up.",
         "title": "Connection Attempts",
         "type": "integer"
      },
      "retry_delay": {
         "default": 5,
         "description": "RabbitMQ retry delay, in seconds.",
         "title": "Retry Delay",
         "type": "integer"
      }
   }
}

Fields:

  • connection_attempts (int)

  • content_type (str)

  • delivery_mode (int)

  • heartbeat (int)

  • host (str)

  • keycloak_authentication (bool)

  • message_expiration (str)

  • port (int)

  • retry_delay (int)

  • tls (bool)

  • virtual_host (str)

field connection_attempts: int = 3

RabbitMQ connection attempts before giving up.

field content_type: str = 'text/plain'

RabbitMQ MIME content type (application/json, text/plain, etc.).

field delivery_mode: int = 2

RabbitMQ delivery mode (1: non-persistent, 2: durable).

field heartbeat: int = 30

RabbitMQ heartbeat interval, in seconds.

field host: str = 'localhost'

RabbitMQ host.

field keycloak_authentication: bool = False

Keycloak authentication for RabbitMQ.

field message_expiration: str = '60000'

RabbitMQ expiration, in milliseconds.

field port: int = 5672

RabbitMQ port.

field retry_delay: int = 5

RabbitMQ retry delay, in seconds.

field tls: bool = False

RabbitMQ TLS/SSL.

field virtual_host: str = '/'

RabbitMQ virtual host.

Example:

servers:
  rabbitmq:
    keycloak_authentication: False
    host: "localhost"
    port: 5672
    tls: False
    virtual_host: "/"
    message_expiration: "60000"
    delivery_mode: 2
    content_type: "text/plain"
    heartbeat: 30
    connection_attempts: 3
    retry_delay: 5

Keycloak Configuration

pydantic model KeycloakConfig

Show JSON schema
{
   "title": "KeycloakConfig",
   "type": "object",
   "properties": {
      "host": {
         "default": "localhost",
         "description": "Keycloak host.",
         "title": "Host",
         "type": "string"
      },
      "port": {
         "default": 8080,
         "description": "Keycloak port.",
         "title": "Port",
         "type": "integer"
      },
      "realm": {
         "default": "master",
         "description": "Keycloak realm.",
         "title": "Realm",
         "type": "string"
      },
      "tls": {
         "default": false,
         "description": "Keycloak TLS/SSL.",
         "title": "Tls",
         "type": "boolean"
      },
      "token_refresh_interval": {
         "default": 60,
         "description": "Keycloak token refresh interval, in seconds.",
         "title": "Token Refresh Interval",
         "type": "integer"
      }
   }
}

Fields:

  • host (str)

  • port (int)

  • realm (str)

  • tls (bool)

  • token_refresh_interval (int)

field host: str = 'localhost'

Keycloak host.

field port: int = 8080

Keycloak port.

field realm: str = 'master'

Keycloak realm.

field tls: bool = False

Keycloak TLS/SSL.

field token_refresh_interval: int = 60

Keycloak token refresh interval, in seconds.

Example:

  keycloak:
    host: "nost.smce.nasa.gov"
    port: 8443
    tls: True
    token_refresh_interval: 10
    realm: "NOS-T"

Execution Section

Defines parameters controlling simulation execution and time management.

General Configuration

Example:

execution:
  general:
    prefix: sos

Manager Configuration

Example:

  manager:
    sim_start_time: "2019-03-01T23:59:59+00:00"
    sim_stop_time: "2019-03-10T23:59:59+00:00"
    start_time:
    time_step: "0:00:01"
    time_scale_factor: 288
    time_scale_updates: []
    time_status_step: "0:00:01" # 1 second * time scale factor
    time_status_init: "2019-03-01T23:59:59+00:00"
    command_lead: "0:00:05"
    required_apps:
      - manager
      - planner
      - appender
      - simulator
    init_retry_delay_s: 5
    init_max_retry: 5
    set_offset: True
    shut_down_when_terminated: False

Managed Application Configuration

Example:

  managed_application:
    time_scale_factor: 288
    time_step: "0:00:01" # 1 second * time scale factor 
    set_offset: True
    time_status_step: "0:00:10" # 10 seconds * time scale factor
    time_status_init: "2019-03-01T00:00:00+00:00"
    shut_down_when_terminated: False
    manager_app_name: "manager"

Channels Section

Defines the messaging channels used for communication between components. This entire section is optional. If a user wants to define each channel and queue for organizational purposes, they can do so here. Otherwise, the NOS-T Tools library will create default channels and queues.

Channels follow this structure:

channels:
  <component>:
    <message_type>:
      address: '<prefix>.<component>.<message_type>'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: <prefix>
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0

Example:

channels:
  satellite: 
    location:
      address: 'sos.constellation.location'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.mode:
      address: 'sos.constellation.status.mode'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.ready:
      address: 'sos.constellation.status.ready'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.time:
      address: 'sos.constellation.status.time'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
  manager:
    init:
      address: 'sos.manager.init'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    start:
      address: 'sos.manager.start'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    stop:
      address: 'sos.manager.stop'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0             
    status.mode:
      address: 'sos.manager.status.mode'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.time:
      address: 'sos.manager.status.time'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0

In this example YAML file, the configuration includes predefined channels for:

  1. Satellite components (location, status.mode, status.ready, status.time)

  2. Manager components (init, start, stop, status.mode, status.time)

Each channel specifies:

  • An address pattern used as the routing key

  • AMQP binding configuration including exchange properties

Using the Configuration File

Applications using the NOS-T Tools library specify the path to the YAML configuration file when initializing. The library reads this file to establish connections to the RabbitMQ broker and Keycloak authentication server and to configure the execution parameters. Refer to Creating a NOS-T Publisher-Consumer Example for an example of how the configuration file is used within the NOS-T Tools library.

Complete Configuration Example

Below is a complete example of a YAML configuration file that can be used with NOS-T Tools:

info:
  title: Novel Observing Strategies Testbed (NOS-T) YAML Configuration
  version: '1.0.0'
  description: Version-controlled configuration file for Snow Observing Systems (SOS) project
servers:
  rabbitmq:
    keycloak_authentication: False
    host: "localhost"
    port: 5672
    tls: False
    virtual_host: "/"
    message_expiration: "60000"
    delivery_mode: 2
    content_type: "text/plain"
    heartbeat: 30
    connection_attempts: 3
    retry_delay: 5
  keycloak:
    host: "nost.smce.nasa.gov"
    port: 8443
    tls: True
    token_refresh_interval: 10
    realm: "NOS-T"
execution:
  general:
    prefix: sos
  manager:
    sim_start_time: "2019-03-01T23:59:59+00:00"
    sim_stop_time: "2019-03-10T23:59:59+00:00"
    start_time:
    time_step: "0:00:01"
    time_scale_factor: 288
    time_scale_updates: []
    time_status_step: "0:00:01" # 1 second * time scale factor
    time_status_init: "2019-03-01T23:59:59+00:00"
    command_lead: "0:00:05"
    required_apps:
      - manager
      - planner
      - appender
      - simulator
    init_retry_delay_s: 5
    init_max_retry: 5
    set_offset: True
    shut_down_when_terminated: False
  managed_application:
    time_scale_factor: 288
    time_step: "0:00:01" # 1 second * time scale factor 
    set_offset: True
    time_status_step: "0:00:10" # 10 seconds * time scale factor
    time_status_init: "2019-03-01T00:00:00+00:00"
    shut_down_when_terminated: False
    manager_app_name: "manager"
channels:
  satellite: 
    location:
      address: 'sos.constellation.location'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.mode:
      address: 'sos.constellation.status.mode'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.ready:
      address: 'sos.constellation.status.ready'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.time:
      address: 'sos.constellation.status.time'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
  manager:
    init:
      address: 'sos.manager.init'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    start:
      address: 'sos.manager.start'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    stop:
      address: 'sos.manager.stop'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0             
    status.mode:
      address: 'sos.manager.status.mode'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0
    status.time:
      address: 'sos.manager.status.time'
      bindings:
        amqp:
          is: routingKey
          exchange:
            name: sos
            type: topic
            durable: false
            autoDelete: true
            vhost: /
          bindingVersion: 0.3.0