Implementation details

To ensure the results can be reproduced, we describe the implementation of our bench-marks in this section. For most benchbench-marks a C version is available in the sources men-tioned in Table 7.1. The sources for heat detection, LEC and outlier detection are not available online, but are listed in the appendices.

The bubble sort, heap sort, FFT, binary search, and outlier detection benchmarks could all be implemented for different data sizes. In this evaluation 16-bit data is used.

8-bit data is too small for many tasks, for example the ATmega’s has 10-bit ADCs, and the memory constraints of sensor nodes mean developers will often be reluctant to use 32-bit variables where 16 bits are sufficient. Therefore, the middle option is used for the main evaluation, and the effect on the performance of 8-bit or 32-bit data is discussed in Section 7.9.

The C versions of these benchmarks were first translated directly to Java, keeping both implementations as close as possible. The result was then manually optimised as described in Section 5.2. These optimisations did not affect the performance of the C version significantly, indicating avr-gcc already does similar transformations on the original code.

The C version was followed as closely as possible to avoid bias by optimising specif-ically for our VM. We take a bit more liberty for the MoteTrack and heat calibration benchmarks, since these could not be directly translated. There are cases where a devel-oper who is aware of the performance characteristics of the VM may choose a different approach than the one used in the C version when directly implementing in Java. We discuss some of the issues when translating C to Java for the CoreMark benchmark in Section 7.2.2, including two extra optimisations that could not be done automatically by an optimising compiler, but do lead to better performance.

The benchmarks exposed some limitations of using a VM instead of native code, which are common to most sensor node VMs. Specifically, the lack of support for constant data, high memory overhead for code containing many small objects, and high performance overhead for allocating temporary objects. These are discussed in more detail in Chapter 8, where we also suggest options to solve some of these limitations.

FFT

Both 8-bit and 16-bit versions of the fix_fft.c implementation exist. In the main evaluation the 16-bit version is used, taken from the Harbor source code [94]. Both ver-sions contain a table of precalculated sine wave values, which are stored in flash using the constant array optimisation introduced in Section 5.3.5.

Outlier detection

The outlier detection algorithm was implemented as described in [49]:

The outlier detector samples a set of sensor values and stores them in a buffer. Once the buffer is filled, it computes the distance between all pairs of samples in the buffer and stores the result in a matrix. Using a distance threshold, the algorithm marks the distance measurements in the matrix that are greater than the threshold. If the majority of the distance measurements for a sensor readings are marked, then the sensor reading is classified as an outlier.

Note that there is no reason to store the distances in a matrix, and having to allocate this matrix limits us to small arrays of input data. The same result can be calculated directly, without the distance matrix, by examining the samples one at a time, and counting the number of other samples with a distance higher than the threshold.

Because this benchmark will be used to compare CapeVM’s safety cost to Harbor’s, it was implemented as described in the paper.

LEC compression

The LEC algorithm is described in detailed pseudo code in [63]. Our implementation follows this pseudo code as closely as possible, and is listed in Appendix A. The input is a set of 256 16-bit ECG measurements downloaded from PhysioNet [71].

MoteTrack

The MoteTrack application uses received signal strength (RSSI) measurements from a number of beacon nodes to do indoor localisation. It contains a database in flash memory of reference RSSI signatures, stored in a complex structure of many small structs and arrays in C.

The memory overhead when translating this directly to Java was too high to run the application, forcing us to make two modifications. First, MoteTrack has the option to store

RSSI signatures at different transmission powers, but the authors note this may not always improve results. The original C code only uses a single transmission power setting, which results in arrays of a single element that get optimised away at compile time. In the Java version these were replaced with simple variables. Second, a two element array with RSSI values for different channels was flattened into two separated variables to eliminate the overhead from allocating too many small arrays.

Again, the constant array optimisation was used to place the data in flash memory.

Without this optimisation, it would be impossible to implement this application in Java because the 20 KB database is too large to fit in RAM. Since this only allows arrays of integer types, the single array of nested C structs was split into 7 arrays for the individual fields.

Thus, while our Java implementation of MoteTrack does execute the same algorithm as the C version, we were forced to modify its implementation significantly, which clearly highlights some of the weaknesses of current sensor node VMs. These changes do not affect the results of the current version of the code, but we note that while it would be possible to use multiple transmission powers or more channels in the C version, this would require too much memory for the Java version.

Heat detection

The heat detection application is adapted from code used by a different project in our group to track persons and fire hazards using Raspberry Pi devices equipped with an 8x8 heat sensor.

It contains two phases: first the heat sensor is calibrated with no heat sources in view to determine the average and standard deviation of the sensor readings. In the next phase the algorithm tracks the position of a person moving within the field of view of the sensor, and detects extreme temperatures that may indicate a fire. In the evaluation both phases are listed separately.

The calibration phase was modified to allow it to run on the more resource-constrained sensor node, but the results of the calibration are identical. The code for the detection

phase was copied directly from the source used on the Raspberry Pi, but modified slightly to avoid repeatedly allocating temporary objects as described in Section 8.8.

Our implementation reads sensor measurements using a native call to read from a table in flash memory, simulating to how a real version would use a library call to read from a sensor.