Organizing Tcl modules for Farfalle

Background

The modeled domains for the Data Aggregator component have been translated using both micca and rosea. The rosea based translation has several purposes:

The entire Data Aggregator can be simulated on a Linux machine which has a network connection and is Bluetooth LE capable.
It is possible to test Data Accessor code using only a local machine.
By replacing selected bridges, it is possible to simulate a Remote Sensor component, again enabling testing with only a single Linux machine.

Use of the Tcl module system

Tcl has two distinct means of finding files that are treated as packages. The older system uses files named pkgIndex.tcl placed in a fixed directory organization. A new module system was created to overcome some limitations of the older pkgIndex.tcl scheme. The tm command manual page gives the details. In summary, modules work as follows:

The module code can be brought into the interpreter using the source command. This includes binary as well as script-based packages.
Meta-data required for the package system is encoded in the file name of the module file.
The search path and directory organization for finding modules is separate and distict from the older pkgIndex scheme.

Module search paths are either from a set of default paths or from a set of paths augmented by the ::tcl::tm::path add command. There are three ways to specify the default search paths for finding modules.

A set of system specific paths whose root is at a directory which is compiled into the interpreter.
A site specific path which has a different naming convention than system specific paths but which has the same directory root as the system specific paths.
A user supplied environment variable.

In addition, we need to deal with the difference between development and deployment locations. For now, we focus on the development locations. Our expectation is that any deployment will be done as a single file executable where all required files are bundled with the executable.

The location for rosea translation modules is:

<...>/Engineering/Software/Phase1/Desktop/DataAggregator/modules

where <...> represents the base directory for the fossil working directory used for development.

As bothersome as environmental variables are to maintain, they provide the mechanism for determining module search paths without modifications to the source of an application or conditional testing within the application startup. So the following environment variable should be set:

export TCL8_6_TM_PATH=<...>/Engineering/Software/Phase1/Desktop/DataAggregator/modules

again, where <...> is replaced by a site specific directory representing a working directory for a fossil checkout of the project.

Modules organization

Each of the pieces of a rosea domain are packaged as Tcl modules. There are multiple modules needed to compose the complete translation.

A module for the domain.
A module for each external entity defined by the domain.
A module which contains the implementation of the semantic mappings between external entities that are client / service pairs.
A module which represents external entities as either non-functioning stubs or simulations of a service provided by an external entity. These stubs are used as part of the testing for the domain translations. Note in the case of stubs, that the namespace where the stub resides must match that of the complete implementation even though the package name is different.

For the Data Aggregator component, the general rules are specifically enumerated as (with x.x specifying the version number of the module).

aggrmgmt-x.x.tm, the Aggregation Management domain
upld-x.x.tm, the Remote Sensor Upload domain
offld-x.x.tm, the Data Accessor Offload domain
SENSOR-x.x.tm, the Aggregation Management SENSOR external entity
UPLOAD-x.x.tm, the Aggregation Management UPLOAD external entity
NOTIFY-x.x.tm, the Remote Sensor Upload NOTIFY external entity
BT-x.x.tm, the Remote Sensor Upload NOTIFY external entity
XFER-x.x.tm, the Data Accessor Offload XFER external entity
NETWORK-x.x.tm, the Data Accessor Offload NETWORK external entity
SENSOR_NOTIFY-x.x.tm, the semantic mapping between the SENSOR and UPLOAD entities
UPLOAD_XFER-x.x.tm, the semantic mapping between the UPLOAD and XFER entities
SENSOR_stub-x.x.tm, a non-functioning stub implmentation for the Aggregation Management SENSOR external entity
UPLOAD_stub-x.x.tm, a non-functioning stub implmentation for the Aggregation Management UPLOAD external entity
NOTIFY_stub-x.x.tm, a non-functioning stub implmentation for for the Remote Sensor Upload NOTIFY external entity
BT_stub-x.x.tm, a non-functioning stub implmentation for for the Remote Sensor Upload NOTIFY external entity
XFER_stub-x.x.tm, a non-functioning stub implmentation for for the Data Accessor Offload XFER external entity
NETWORK_stub-x.x.tm, a non-functioning stub implmentation for for the Data Accessor Offload NETWORK external entity

The layout of the module files in the file system appears as:

modules
├── aggrmgmt
│   ├── SENSOR-1.0.tm
│   ├── SENSOR_stub-1.0.tm
│   ├── UPLOAD-1.0.tm
│   └── UPLOAD_stub-1.0.tm
├── aggrmgmt-1.0.tm
├── bridge
│   ├── SENSOR_NOTIFY-1.0.tm
│   └── UPLOAD_XFER-1.0.tm
├── offld
│   ├── NETWORK-1.0.tm
│   ├── NETWORK_stub.1.0.tm
│   ├── XFER-1.0.tm
│   └── XFER_stub.1.0.tm
├── offld-1.0.tm
├── upld
│   ├── BT-1.0.tm
│   ├── BT_stub-1.0.tm
│   ├── NOTIFY-1.0.tm
│   └── NOTIFY_stub-1.0.tm
└── upld-1.0.tm

Composing domains into a script

Each script that exercises one or more domains must include the packages needed for the domain any any of its external entities. We would like to specify the packages simply but with some flexiblity.

For example, a single domain integration test for the Aggregate Management domain can be scripted as:

package require aggrmgmt
package require aggrmgmt::SENSOR_stub
package require aggrmgmt::UPLOAD_stub

A complete Data Aggregator application script can be obtained by:

package require aggrmgmt
package require aggrmgmt::SENSOR
package require aggrmgmt::UPLOAD

package require upld
package require upld::NOTIFY
package require upld::BT

package require offld
package require offld::XFER
package require offld::NETWORK

Two things of note:

To provide flexibility in composing scripts, the domain package does not package require any of its external entity packages. This allows us to use complete implementations or stubbed implementations depending upon the application script we are composing.

For complete external entity implementations, they do package require the necessary semantic mapping code, e.g.

package require bridge::SENSOR_NOTIFY

would be contained in the aggrmgmt::SENSOR and upld::NOTIFY packages.

== Namespace considerations

Tcl code resides in a namespace. Although there is no language binding between namespace names and module names, long practices has shown that keeping the two names the same is a good practice to reduce confusion. So we expect the following of rosea translated packages.

The code for a domain resides in a namespace of the same name, i.e. the Aggregation Management domain resides in the aggrmgmt namespace.
Namespace names should be fully qualified (there are a few exceptions to this, but each requires careful consideration).
An ensemble command should be generated whose name is the same as the namespace and whose subcommands are the externally invocable interface.
External entity packages which contain stub implementation must use the namespace of the complete implementation. In this case, the package name and namespace name are different.

Note that rosea does handles these conditions for translated domains automatically. For external entities, this must be provided as part of the implementation. For example, the SENSOR external entity would have the equivalent of:

package require bridge::SENSOR_NOTIFY

namespace eval ::aggrmgmt::SENSOR {
    namespace export Connect

    proc Connect {sensorID} {
    }

    # ... other exports and proc definitions

    namespace ensemble create
}
package provide aggrmgmt::SENSOR 1.0