0 200 400 600 800 1000 1200 1400
1 2 4 8 16
Number of Client Processes
Throughput (requests/sec)
K7_LOCAL K7_REMOTE PII_LOCAL PII_REMOTE
For the last two decades, many research efforts on extensible systems have been proposed in the literature. However, such systems usually require much more resources. None of them addressed the problem of achieving the extensibility on resource-constrained embedded devices. Moreover, some of them result in the degradation of the system performance. In the following, we describe these research efforts. To make the description more clear, we divide some of the research efforts into several categories: micro-kernel, extensible kernel, Java operating system, and loadable kernel module (LKM). Other efforts are described following the description of the efforts in the above categories.
5.1 Micro-Kernel
Research on micro-kernels (Rashid et al., 1989a; Rashid et al., 1989b; Zuberi and Shin, 1996) moves most of the traditional operating system functionality to user-level server processes. It simplifies the extension of the operating systems since the extension can be achieved by adding or replacing the server processes. However, it suffers from the performance problem. Since most operating system services are implemented in user level, a service request requires two more context switches and another pair of protection-domain crossings. Moreover, micro-kernels and traditional embedded kernels are totally different in the kernel structure. Therefore, extending embedded kernels with this approach involves kernel re-design.
In contrast with the micro-kernel approach, we focus on making an existing embedded kernel become extensible without re-designing the kernel. In addition, we use kernel-level modules so that our system will not suffer from the performance problem. Finally, we make use of the server-side module linking to reduce the resource requirements (specifically, the memory footprints) of the client kernels.
5.2 Extensible Kernel
To address the performance problem of micro-kernels, extensible kernels allow user applications to inject codes into the kernels so as to extend them. Because the injected codes run in the kernel mode, extensible kernels usually result in better performance than that of micro-kernels. However, the injected codes usually can not run in full speed because of the security problems. Extensible kernels often perform run-time checks to ensure that the injected codes will not damage the kernels. For example, SPIN (Bershad et al., 1995) uses software fault isolation (SFI) (Wahbe et al., 1993) to restrict the memory area that the injected codes can access. Systems such as packet filters (Mogul et al., 1987), and HiPEC (Lee et al., 1994) allow interpretative codes to be injected and use kernel mode interpreters to enforce the system security.
Such run-time checks do degrade the system performance.
The main reason of the performance problem is that the kernels do not trust the extension codes. In contrast with their approaches, we assume that kernel extensions (i.e., kernel modules) are trusted. This assumption is used in the previous research (Auslander et al., 1997). It also holds for most of the UNIX implementations. In our approach, kernel modules are developed by trusted system programmers or third parties, and are verified prior to execution. Therefore, they can run in full speed. In addition, our approach requires no re-design of the operating systems and puts efforts on reducing the resource requirements of the kernels.
5.3 Java Operating System
Java operating systems (JOS, 1997; Saulpaugh et al., 1999) allow the programmers to write system modules such as TCP/IP, and file systems in Java (Ritchie, 1997). The system modules are compiled as Java class files and are loaded by the JVM at run time. Liao et al. (Liao et al., 1996) take another approach. They insert a JVM into a micro-kernel so as to allow users to write Java programs to extend the kernel. Similar to the previous techniques, these approaches also suffer from performance problems. The major problem is that Java codes are interpreted, not compiled. Although the performance of interpretation improves since the birth of Java, there is still a performance gap between the interpreted and the compiled codes. Some Java platforms take advantage of just-in-time compilation (JIT) to improve the performance. This technique, however, is hard to be applied on resource-limited devices since it consumes too much memory to perform the compilation. In addition, the codes generated by the JIT compilation are less optimized since there is not much time to perform code optimizations.
In contrast with the Java operating systems, we use compiled codes. Since codes are compiled off-line, there is much time to perform optimizations. In addition, we focus on making a C language-based embedded kernel extensible instead of re-writing a whole kernel in Java. And finally, we put efforts on reducing the resource requirements of the kernels.
5.4 Loadable Kernel Module (LKM)
As described earlier, many desktop operating systems such as Linux, provide loadable kernel modules to extend their kernels at run time. After being installed, kernel modules can run in full speed without any further run-time checks. The major problem of the LKM is its space overheads. As shown in the performance measurement section, it requires much more overheads than our approach.
Oikawa et al. (Oikawa et al., 1996) take an approach similar to the LKM. They make RT-Mach (Tokuda et al., 1990) extensible by introducing Dynamic Kernel
Modules (DKMs). The same as LKMs, DKMs are stored as files and are able to be loaded/unloaded at run-time. However, DKMs are managed by a user-level DKM server, instead of the kernel. The server is responsible for loading a DKM when it is needed, and unloading a DKM when the kernel is short of memory. In contrast to this approach, we manage the kernel modules at remote site, reducing much local memory requirements. In addition, we address the issue of run-time module replacement, which is not mentioned in their work.
In addition to the research described above, there are still many efforts on system extensibility or customizability. At the end of this section, we give brief descriptions on them.
Kohler et al. (Kohler et al., 2000) propose a configurable IP router architecture named Click. Under this architecture, router functions are implemented in elements, which can finally be composed into a single Linux kernel module. By composing different elements, the developers can customize the router according to their needs.
However, this customization is too coarse-grained even that it does support fine-grained components (i.e., elements). This happens because all elements are finally linked into a single module, and therefore the customization requires the whole router to be unloaded before the new router can be loaded. Router Plugins (Decasper et al., 2000) is a software architecture for fine-grained customization of routers. It uses the LKM mechanism supported by the kernels to load different plugins to extend the functionality of the router. Since routers are usually equipped with rich resources, the researchers did not address the resource-limited problems that we encountered.
The OSKit (Ford et al., 1997) allows users to build their customized kernels by composing different system components. It also has a linking toolkit, named Knit (Reid et al., 2000), to assist users when composing these components. Different from our work, the researchers put most of their efforts on the link-time techniques, instead of providing a customizable and economic run-time environment.
Helander and Forin (Helander and Forin, 1998) propose a modular system architecture. They also describe a dynamic module-updating mechanism that is similar to our module replacement technique. However, in their architecture, the extension modules are loaded via the file system. Therefore, it is similar to the LKM approach that we mentioned above. As we described in the performance measurement section, our approach requires much less overheads than theirs.
DEIMOS (Clarke and Coulson, 1998), Kea (Veitch and Hutchinson, 1996), and Pebble (Gabber et al., 1999) are all extensible systems. However, they maintain symbol tables on the local site for dynamic extension. In addition, they do not address the problem of run-time module replacement.
Finally, some other research (Gheith et al., 1994; Messer and Wilkinson, 1996)
allows the internal implementation, or the invocation methods of a service to be varied at run time, according to the invocation attributes. However, since all of the possible implementations of a service have to be installed in prior, this approach does not consider the resource consumption of the kernels.
6. Limitations
Although the OSP framework provides a way to extend embedded kernels effectively. It does have some limitations. In this section, we will describe these limitations.
Scalability: In our prototype implementation, we use single sever machine as the OS Portal. Moreover, the OS Portal maintains a persistent connection for each on-line client. Therefore, the average request latency for a client will become longer as the number of on-line client increases. According to our experiment, the request latency is proportional to the number of on-line clients. When there are 100 clients, the request latency is 0.124 seconds while it becomes 1.26 seconds as the client number reaches to 1000. The experiment reveals that the request latency will become unacceptable if there are large number of clients, say 10000. We intend to use the following approaches to increase the scalability of the framework. First, we will replace the single server machine with a cluster of servers. Second, we intend to use connectionless protocols for client-server communication to reduce the server overheads for each client. In the future we will implement the approaches and evaluate the result performance.
Static Module Interfaces: Although the client kernels can dynamically download kernel modules to extend their functionality, module interfaces are static. They must be defined and implemented in the client kernels in advance. For example, if a client kernel doesn’t implement a scheduler interface, it cannot perform module replacement on scheduler modules. To enable the replacement, a scheduler interface must be incorporated into the kernel and therefore kernel modification is needed.