The IoTtalk is defined in two domains: device domain and network domain. The architecture of IoTtalk is shown in Figure 2.5.
Figure 2. 5 The IoTtalk Architecture
In the device domain, an IoTtalk device consists of two software components, Sensor and Actuator Application (SA) and Device Application (DA). The SA (Figure 2.5 (a)) implements the software executed in the IoT hardware device. It is responsible for implementing the IoT device functions such as the PM2.5 algorithm or the light intensity and color circuit software. The DA (Figure 2.5 (b)) is responsible for communications with the
14
network domain in the IoTtalk system. It mainly handles the registration process and data exchange via MQTT protocol carried by wired or wireless (i.e., Wi-Fi, 3G, or LTE) media.
While an IoT device attaches to the IoTtalk, the DA initiates a registration procedure to inform the IoTtalk server of the new connection. After registering successfully, the DA can start transmitting and receiving data. There are two categories of device applications, according to how we create them. The AG devices (Figure 2.5 (a) and (b)) can be automatically created by the AutoGen Subsystem (Figure 2.5 (j)), whereas the manual devices (Figure 2.5 (c) and (d)) should be created manually by users.
The network domain in IoTtalk is a well-modularized cloud server that manages network applications for IoTtalk devices. It consists of several subsystems and graphical user interfaces. The Creation, Configuration, and Management Subsystem (CCM; Figure 2.5 (e)) manages the features of IoT devices. It also automatically configures the connection between IDFs and ODFs. The configuration generated by CCM Subsystem will be stored in the IoTtalk database system (IoTtalk DB; Figure 2.5 (f)). The Execution and Control Subsystem (EC; Figure 2.5 (g)) is the core of the whole system. It is responsible for the control plane and the data plane of the connection between IoTtalk devices and the server. For the control plane, there are APIs implemented based on the MQTT protocol. DAs publish/subscribe to specific topics to send/retrieve the control messages for their DFs. In addition, if an IoT device connects/disconnects to/from IoTtalk, its DA informs EC to change its status to online or offline via the MQTT APIs. As for the data plane in EC, NAs will be created and executed for the connected IDFs and ODFs. The NAs first process the data received from IDFs with some functions or algorithms (e.g., normalization) and then deliver them to the connected ODFs.
15
The IoTtalk Graphical User Interface (IoTtalk GUI, Figure 2.5 (h)) provides a friendly web-based user interface to quickly establish connections and meaningful interactions among the IoT devices. Figure 2.6 illustrates an example of the IoTtalk GUI. On the top of the interface window is a menu bar (Figure 2.6 (a)) with several functions (Figure 2.6 (b)-(i)).
Users can switch between IoTtalk projects by clicking the “Project” item (Figure 2.6 (a)). The
“Model” item is a drop-down list of device models. By selecting the DMs with their corresponding DFs, users may add the chosen DMs to the current projects. For example, in this project, the “Smartphone” and “Bulb” device models (Figure 2.6 (j) and (k)) are added.
The “Flush” item (Figure 2.6 (d)) is a toggle button to flush the states of the project when IoTtalk suspends the project due to wrong settings. The color of the “Flush” item indicates the status (green for "on", red for "off"). If a user clicks the green “Flush” button, the status will change to "off." After 5 seconds later, IoTtalk will automatically alter the status to "on" to complete the process of flushing. Users may press the “Delete” item (Figure 2.6 (e)) to delete the current IoTtalk project. To activate or deactivate the simulation function, users can toggle the “Simulation” item. After the activation, a simulator simulates a real IoT device and starts sending data. Users can effectively verify the correctness of their NA settings during the simulation. The “Import” item (Figure 2.6 (g)) and “Export” item (Figure 2.6 (h)) is used to import/export an IoTtalk project from/to a text file. The “Logout” item (Figure 2.6 (i)) is a button to sign out the current user.
16
Figure 2. 6 The IoTtalk GUI
The Graphical Layout Window (Figure 2.6 (j)) graphically shows the IoT devices and their connections. The graphical representation of an IoT device is called Device Object (DO) (see Figure 2.6 (k) and (l)). A device object is the graphical representation for an instance of the device models created in the GUI and can be mapped to a real IoT device. The device object consists of a device setting object (Figure 2.6 (m) and (n)), a device model name object (Figure 2.6 (o) and (p)), and several device feature (DF) objects (Figure 2.6 (q) and (r)). By clicking the setting object, the user can set up the parameters, save or delete the device object.
To attach real devices to the device objects, one can press the name objects. For connecting IDF objects with ODF objects, users can toggle the DF objects. All connections (Figure 2.6 (s)) specified in the Graphical Layout Window provide the data paths such that the data are transferred from the IDFs to the ODFs through these connections. The connections can be created by clicking a device feature object of an input device object followed by clicking the other one in the output device object.
17
The Account Subsystem (Figure 2.5 (i)) is a standalone system that manages the creation, deletion, and authentication of accounts for IoTtalk. To achieve the single sign-on (SSO) concept, the Account Subsystem provides OAuth [12] for access delegation. With the OAuth service, ScratchTalk Subsystem can effectively speed up the user registration and sign-in process. The Account Subsystem Graphical User Interface (Figure 2.5 (j)) is the web-based graphical interface for users to manage their accounts. The Authentication and Authorization Subsystem (AA; Figure 2.5 (k)) is responsible for managing users’ access to IoTtalk. While users attempt to manipulate the system, AA Subsystem checks if they have been authenticated and authorized. Every authorized user possesses a valid identity given by the AA Subsystem for future access in the IoTtalk system.
The AutoGen Subsystem (Figure 2.5 (n)) effectively integrates IoT with ScratchTalk Subsystem (Figure 2.5 (l)) by automatically generating devices and projects. Users can interact with their Scratch application through smartphones instantly.
The ScratchTalk Subsystem (Figure 2.5 (l)) is an application developed based on IoTtalk.
ScratchTalk is associated with a graphical user interface (ScratchTalk GUI; Figure 2.5 (m)) modified from the Scratch (Figure 2.1). In the ScratchTalk GUI, users can make creative Scratch projects with the functional blocks and interact with real IoT devices. Figure 2.7 shows an example of sending sensor data to ScratchTalk. Figure 2.7 (a) shows an area in the GUI called “stage.” It displays the visualized Scratch programs. In this Scratch project, users can find the variables shown in Figure 2.7 (b) changes according to the data captured by the smartphone (Figure 2.7 (c)). Tilting the smartphone in specific directions determines the
“Horizontal” and “Vertical” value. On the other hand, the “Volume” value indicates the
18
volume of voice received by the smartphone.
Figure 2. 7 Displaying sensor data in ScratchTalk project
When a user attempts to make a Scratch project to interact with real devices, they simply add the Scratch extension for IoTtalk (described in Section 3.2) to their Scratch project.
The ScratchTalk Subsystem (Figure 2.5 (l)) then informs AutoGen Subsystem to create an IoTtalk project for the Scratch project. Figure 2.8 illustrates the IoTtalk project configuration for a Scratch project. Figure 2.8 shows the IoTtalk project configuration for a Scratch project.
The “Smartphone” (Figure 2.8 (a)) is the input device model which implements the DA/SA for the smartphone device. The output device model “Scratch” (Figure 2.8 (b)) implements the DA/SA for the Scratch application. The Smartphone model has two IDFs, including Orientation-I and Microphone-I. Similarly, the Scratch model is composed of three ODFs:
Horizontal-O, Vertical-O, and Volume-O. Join 1 and Join 2 links handle the transmission of the orientation and microphone data, respectively. After receiving values, the SA for the Scratch can take action according to the program created by users.
19
Figure 2. 8 The IoTtalk project for a Scratch project