國立交通大學
電機資訊國際學位學程
碩士論文
Integration and interoperability in the context of enabling
end-user to incorporate arbitrary electronic devices and software
into a collective population of resources capable of cooperation.
研究生：Krzysztof Kamil Jacewicz, 秦學思
指導教授：黃經堯 博士
Integration and interoperability in the context of enabling
end-user to incorporate arbitrary electronic devices and software
into a collective population of resources capable of cooperation.
研究生：秦學思
Student：Krzysztof K. Jacewicz
指導教授：黃經堯 博士
Advisor：Dr. ChingYao Huang
國立交通大學
電機資訊國際學位學程
碩士論文
A Thesis
Submitted to EECS International Graduate Program National Chiao Tung University
in Partial Fulfilment of the Requirements for the Degree of
Master
January 2013
Hsinchu, Taiwan, Republic of China 中華民國一〇二年一月
National Chiao Tung University
Authorization of Copyright to E-version of
Doctoral/Master ’s Thesis
(For licensor to bind as 2nd page of the thesis’s inside cover)
The degree thesis authorized is required by _______________
(Dept.), National Chiao Tung University in _____
(Month), ______
(Year).
Thesis Title:
Integration and interoperability in the context of enabling end-user to
incorporate arbitrary electronic devices and software into a collective
population of resources capable of cooperation.
Advisor: Dr ChingYao Huang, 黃經堯 博士
■ Agree □ Disagree
I hereby, non-exclusively and for free, authorize this thesis to
National Chiao Tung University and University System of Taiwan
Library. Based on promoting the idea of ‘Resource Sharing & Mutual
Benefit and Collaboration’ among readers, as well as feed back to the
society and academic research, National Chiao Tung University and
UST Library are allowed to include, reproduce and utilize in different
forms (paper, CD-ROM, other digitized materials) regardless of
regions, time, and frequency. Abiding by the Copyright Law, readers
can search online, read, download, or print out the thesis.
Range and Time for Uploading:
NCTU and UST LAN
■ Accessible from /Y. /M. /D.
Off-Campus Internet
■ Accessible from /Y. /M. /D.
Authorized by:
Signature: ______________________
National Chiao Tung University
Authorization of Copyright to
Doctoral/Master ’s Thesis
(For licensor to bind as 2nd page of the thesis’s inside cover)
The degree thesis authorized is required by ________________
(Dept.), National Chiao Tung University in _____
(Month), ______
(Year).
Thesis Title:
Integration and interoperability in the context of enabling end-user to
incorporate arbitrary electronic devices and software into a collective
population of resources capable of cooperation.
Advisor: Dr ChingYao Huang, 黃經堯 博士
■ Agree
I hereby, non-exclusively and for free, authorize this thesis to
National Chiao Tung University. Based on promoting the idea of
‘Resource Sharing & Mutual Benefit and Collaboration’ among
readers, as well as feedback to society and academic research,
National Chiao Tung University is allowed to include, reproduce and
utilize in paper. Abiding by the Copyright Law, readers can read or
print out the thesis.
The thesis accounts for one submitted by me to the MOEA’s
Intellectual Property Office for the patent application (Ignore if not
applying). The application number is ____________________,
please
postpone
the
disclosure
of
the
thesis
to
____/Year____/Month____/Date.
Authorized by:
Signature: ______________________
Authorization of Online Access to
Doctoral/Master’s Thesis, National
Central Library
ID: GT00 (+ Student Number)
The degree thesis authorized is required by ____________________
(Dept.), National Chiao Tung University in _____
(Month), ______
(Year).
Thesis Title:
Integration and interoperability in the context of enabling end-user to
incorporate arbitrary electronic devices and software into a collective
population of resources capable of cooperation.
Advisor: Dr ChingYao Huang, 黃經堯 博士
I hereby agree to non-exclusively and for free authorize this thesis
(Abstract included) to National Central Library, who is allowed to,
regardless to regions, time, and frequency, reproduce in different forms
(MF, CD-ROM, other digitized products) and upload to the Internet for
readers to non-profit-making search online, read, download, or print out
the thesis.
※ The non-profit-making purpose of online search, reading, downloading, or printing should follow the copyright laws and regulations.
Authorized by:
Signature: _______________
NATIONAL CHIAO TUNG UNIVERSITY EECS International Institute
Master’s thesis
Krzysztof Kamil Jacewicz
ABSTRACT 20.01.2013
ABSTRACT
Objectives of the study
How to enhance End-users’ experience by developing a system (and a standard) for integrating owned devices and programs into a network of resources, and be able to selectively and remotely access functions distributed across the network? And how to enable developers to easily make their existing products compatible with the system?
Academic background and methodology
Author’s academic background is in EECS with the concentration in Networking and Telecommunication, and additional interest in Compiler Theory and programming for embedded systems. Professional background includes work at Rawspeed (Beijing) Technology Ltd., as Product Manager for CDN related products, at Vale (Beijing) Internet Technology Ltd., as Product Manager web service and user experience, at Rulingcom Digital Inc. (Hsinchu) as Project Manager, working on CMS system, GIGABYTE Technology (Xindian), working as Account Manager for Channel sales division GIGAZONE, and at American Megatrends Inc. Taiwan Branch (Taipei), as Marketing Specialist and supporting engineer working on projects directly related to cloud computing, remote server management, and web service architecture. Methodology used in the process of research includes review of publications and knowledge acquired during professional work.
Findings and conclusions
In process of the research architecture has been designed and proposed for integration of hardware and software resources. Key finding was that it can be implemented, and even used in a commercial project with much success. It was also found that created solution has saved time and complexity when adding RPC functionality to an existing system and did not require the existing code (originally the eventuality of incorporating ignoring RPC support) to be redesigned in any bit. As a result, the system was found to enable SOA as a Service
Acknowledgments
Foremost, I would like to express my sincere gratitude to my advisor Prof. ChingYao Huang for the continuous support of my study and research, for his immense patience, his motivation,
enthusiasm, and given trust. His guidance helped me in all the time of research and writing of this thesis, with lot of understanding received from his side.
Besides my advisor, I would like to thank the rest of my thesis committee: Dr. Kai-ten Feng,
and Dr. Sheau-Ling Hsieh, for their encouragement, insightful comments, and hard questions.
I want to add special thanks to Allen Chang, the founder and owner of Rulingcom Digital Inc., where I have worked when I first came to Taiwan. Allen himself encouraged me to follow
master degree at the National Chiao Tung University (he is the alumni himself as well) and gave me the flexibility needed for working and studying both in full-time mode.
Last but not the least, I would like to thank my family: my parents Ryszard and Wanda, for
Table of context
List of abbreviations ... I List of appendices ... I
1 Introduction: context to the thesis study ... 1
1.1 Mobile computing ... 1
1.2 Cloud computing ... 2
1.3 “… as a Service”... 3
1.4 Internet Of Things (IOT) ... 5
1.5 Machine-to-machine (M2M) communication ... 5
1.6 Modern System Integration (SI) ... 6
1.6.1 Home automation ... 7
1.6.2 Sensor networks ... 7
1.6.3 High level development tools ... 8
2 Problem analysis ... 9
2.1 Problem definition ... 9
2.2 Current designs and solutions potentially useful to address the problem... 12
2.3 Defining targets for potential solution ... 14
2.3.1 Value to users ... 14
2.3.2 Value to businesses ... 16
2.3.3 Value to industry ... 17
2.3.4 Value to developers ... 20
2.4 Designing benchmarking criteria... 22
2.4.1 Objectives ... 22
2.4.2 Factor 1: Simplicity to End-users ... 23
2.4.3 Factor 2: Simplicity to developers ... 23
2.4.4 Factor 3: Price effectiveness ... 25
2.4.5 Justifying the selection of factors ... 26
2.4.6 Benchmark summary ... 31
3 Solution proposal ... 31
3.1 Overview ... 31
3.2 Layer 1: Distributed API ... 32
3.2.1 Components of the system ... 32
3.2.2 API Operator ... 33
3.2.3 API Contributor ... 34
3.2.4 Buffered RPC protocol ... 36
3.2.5 Self-hosted API Operator ... 36
3.2.6 Virtual API Contributor ... 36
3.2.7 API Contributor integration to existing code ... 39
3.2.9 Custom GUI skins ... 43
3.2.10 API Contributor drivers – the unique feature claim ... 43
3.3 Layer 2: Low-level EUD ... 45
3.4 Layer 3: Semantic Channel... 46
3.5 Layer 4: “Company” model ... 49
3.6 Layer 5: High-level EUD ... 52
3.7 Current implementation ... 53 3.7.1 libEngine.pas ... 54 3.7.2 libSupportedTypes.pas ... 55 3.7.3 libHostedVariables.pas ... 56 3.7.4 libHostedFunctions.pas ... 57 3.7.5 libErrors.pas ... 58
3.7.6 Server/client vs. Operator/Contributor vs. Flavors ... 58
3.7.7 libServerMod_HTTP.pas as a flavor of API Operator ... 60
3.7.8 libMemberTCPLite.pas as a flavor of API Contributor ... 61
3.7.9 Extending flavors ... 61
3.7.10 uAnLex.pas ... 61
3.8 Non-standard approach to RPC ... 62
4 Solution evaluation ... 64
4.1 Addressing defined targets ... 64
4.2 Meeting benchmarking criteria ... 68
4.3 Proof of concept ... 70
4.3.1 Early prototype ... 70
4.3.2 Life demo ... 70
4.3.3 Case study ... 80
4.4 Study of a prior art ... 82
4.4.1 Case study ... 82
4.4.2 Comparison of main differences ... 83
5 Conclusion ... 85 6 Bibliography ... 88 7 Apendices ... 90 7.1 Apendix 1 ... 90 7.2 Apendix 2 ... 91 8 Resume and CV ... 92
I
List of abbreviations
API = Application Programming Interface
IOT = Internet Of Things
EUD = End User Development
M2M = Machine-to-machine (Communication)
RPC = Remote Procedure Call
RMI = Remote Method Invocation
SI = System Integrator
List of appendices
Appendix 1: Screenshot of a Layer 2 GUI via web browser Appendix 2: Listing of attached CD
1
1 Introduction: context to the thesis study
This chapter will selectively highlight few areas of interest significantly related to the matter
of this paper. Before the actual problem is defined in the next chapter, here is a brief introduction to the greater context from which the study has begun.
1.1 Mobile computing
One of the earliest origins of the idea for this research was my first study of mobile computing after I got my first ever mobile device – a HP Jornada handheld PC
running on Windows CE (now discontinued). I had a special interest in developing my own software to run on the mobile device, and later, another strong interest in
remotely communicating from the mobile device to the PC and controlling some features of the server application. My first finding was, that it was much more
complex to develop for mobile, due to things like SDK, cross-compiling, emulating, debugging on a device, and few others. Also, there were no tools for native
development on the mobile device at the time. Even now in 2013 this is still not common. I am using Pepe Le Compiler on my Android handset to natively compile
Pascal code on the device and run it.
My second finding was that it was quite time consuming to implement remote
communication between my mobile device and my PC. These were two different platforms, and I had to re-learn socket programming in another programming
language, that was supported in the SDK for the mobile device. As an effect, I had to write conceptually the same code twice, in two programming languages (at least).
These two findings have eventually made me more appreciative toward web programming. As long as I wanted to use my Smartphone as a remote control for a
2
going through development of a native (or Java) application. Later, with the arrival of HTML5 and some support for accessing in-phone API via webAPI, I started to
appreciate HTTP protocol even more.
1.2 Cloud computing
My next important study that helped me to approach the research covered here, was
in Cloud Computing. The theory that this study covers is very wide, but my special interest was in web service architecture and user experience.
Web API concept felt in sync with my approach to solve some problems in mobile computing. And as soon after first public cloud services started to be available to
end users, I immediately saw an opportunity to help my programming to incorporate these to save lot of coding on information sharing between applications
in a network. Cloud storage has rapidly become one of my favorite cloud services. Before the era of Dropbox, I would either need to remotely connect to a database
(which was sometimes too much overhead for small apps) or to implement file transfer functionality. Today, I could re-write lots of my old applications in a
fraction of time originally spent on them, as I would totally ignore the portion related to data transfer over network. One of my most recent commercial projects
was a digital signage player. The player would keep digital photographs in a local folder, and play them on the screen in an infinite loop. If, at any time, the content of
the directory changed, the player would list all the files again and restart the loop. No network programming included. However, my customer hoped, he could
manage the content remotely from PC in his home, or even better, via mobile phone. He even requested a feature of being able to upload a photo taken with his mobile
phone directly to any Digital Sign in his network. I have Upgraded my player without actually upgrading it. I enabled requested functionality to my customer
3
without changing a line of code in my project. I have set up a Dropbox account for my customer, and installed a client both on his Smartphone and each PC running the
player. I have synced the folder with photos on every player with Dropbox. And that’s it! My customer can take a photo with his mobile phone, and upload it to
Dropbox. The player on the PC will see that the content of the local folder changed as soon as Dropbox syncs it with the cloud. And few moments later, the uploaded photo will display on the Digital Sign. That is my idea of so called “Integration” –
customizing systems to one’s needs, with minimum effort, by integrating resources
designed to share functionality between each other.
On the user experience side, that I mentioned was my other major interest, I found it
almost beautiful, how transparent the whole network programming can be to the end user. I am a huge fan of simple, intuitive interfaces that hide all the complexity from the user’s experience.
1.3
“… as a Service”
When I was working for American Trends, I was assign to work on a business plan
with my colleague with MBA background. The business plan consisted of lot of technology-related problems to address. A lot of it was related to cloud computing,
and particularly to web services, and SaaS (Software-as-a-Service) business model. Soon, also IaaS (Infrastructure-as-a-Service), PaaS (Platform…), NaaS (Network…) and all other “aaS’es” came after. Other than the fact that these are all very hot
topics on the market these days, it helped my view on the integration problem to
evolve to a whole new level. My primary programming language was always Pascal. After I started developing for mobile devices, I had to learn few more programming
languages. Even my rusty C++ ability was of no use (except for developing native ARM executables, in which case either Pascal or C++ were both fine). I have learnt
4
a bit of Java, and Java-based languages, a bit of objective C used for developing for iOS, and learnt few different SDK’s (WindowsCE, Android, iOS, Symbian, BREW),
programming environments/editors, and emulators. There were quite a lot of new things to learn each time before I was able to start coding, and then some more way
to go before being able to compile, and a bit more way to go before I was able to run. The “…as a Service” approach has changed my life for better! Eventually I
came up with my own, alternative way of programming for mobile: if I needed to control an asset on my mobile device, I looked up snippets of code in the Internet to
learn the reading from and writing to the asset and enclose it in few function calls. Then I use previously developed generic code, to export these few functions as a
web service. Now I can immediately use them from any application. Instead of continue to use programming language I am not fluent in, I use Pascal to write an
application calling the web API. Or I use HTML and Javascript to create a static page, with an interface to call the web API. And one of my most original ways to
use this approach, is to design and put a static HTML page on my phone, that I will use to control assets on my phone via phone’s web browser, rather than via mobile
app. If I want to change the program, I only need to change the HTML and Javascript. No compiling needed. And I am not claiming that this way is optimal for
local on-device use, but it is simpler for me than programming in, say, Java, and gives me lot of additional benefits:
- such code is web-service oriented and thus my app can have much broader context of use
- Once exported functions controlling on-device assets, I am no more bound to use standard and troublesome software development process for Android: no
5
At this stage a concept of the later research was already slowly emerging in my mind. I learnt to see how much space there is to come up with new, alternative ways
of doing things. And I started to understand that integrating components into a bigger system saves lot of time otherwise spent on re-inventing the wheel.
1.4 Internet Of Things (IOT)
The first time I really got to know about the IOT was when my thesis adviser Prof. Huang assigned me to work with one of his research teams. Initially I was seeking a
subject for research around sensor networks. I thought that I might use semantic web approach to solve some problems in the area. The Semantic Web was a topic I have
studies a bit while working at Rulingcom Digital Inc., and at the time it was because everyone on the market was discussing Web 2.0 and we wanted to go ahead into the
future, looking for some inspiration for later RND. I thought that Semantic Web can also help in addressing some of the problems of the sensor networks, related to Data
interoperability. Little did I know, that Semantic Web gives much more potential help than only dealing with data. My study into IOT helped me to see, that there is a huge
potential in using IOT to integrate devices and programs. I found out, that developing for the IOT was very challenging and complex problem. Lot of tools was still under
development, or only existed as reference designs. Lot of the topics was still in the stage of theory. But going through lot of work, IOT was offering a chance for ending
up with a solution giving super-intuitive, simplified and interactive user experience. That was just what I was always after – hiding all the complexity from the user
experience and providing him with pure functionality, dressed in an intuitive interface.
1.5 Machine-to-machine (M2M) communication
One of the sub-topics in the IOT is machine-to-machine communication. And this area
6
and businesses. The topic is very broad, but again, I mention it in the context of my research only. Briefly, I got the first idea of the integration system architecture during
the study over M2M. I liked the concept of machines describing their assets to other machines, and enabling mutual collaboration on achieving common goals. I thought that it’s great that M2M is getting more and more popular, because soon it will help
users to integrate their devices in their interactive environments, like homes, offices,
and public venues. But as of the present day, I am not satisfied with available M2M solutions. Although current systems help to solve lot of problems, they just don’t seem
to be solving mine. I am not yet in the stage where I care for the power efficiency management in my home, nor do I find it fancy to deploy m2m system to help me
turning lights off when I forget to do so. With the current pricing the later would rather cause me to loose money than earn on return of investment. I am not saying these are
not practical use cases, I am just saying that these are not my immediate needs. Instead, I would like to have m2m system, that doesn’t require me to purchase expensive
hardware, nor to have a service officer to help me to deploy it. A system, that would allow me to integrate devices in my house to my liking. I want a system that is
lightweight, elastic, an in open architecture, so I can build my custom solutions on top of it myself. And I believe that there are more people like me. Yet, I am not able to
find a suitable solution to address my needs. So instead, I eventually came up with my own, and I want to propose it to others as well.
1.6 Modern System Integration (SI)
After I had a clear idea on what I want to do, I needed to start from researching the prior art. I have studied into few areas that cover the problem of integration on
7
1.6.1
Home automation
The Smart Houses get quite a lot of buzz in the Internet these days, and living in Hsinchu where the Science Park is located allowed me to see that it’s not only the thing that people discuss about. Lot of Tech
companies are already providing solutions in that domain, or working on
such solutions. What I have also observed is that there is much more focus on the Smart House appliances (hardware) than on Smart House
frameworks (software). At least based on my observation of what solutions Taiwanese manufacturers are presenting to the market. And
Taiwanese manufacturers are known to be powering a lot of worldwide technological advancement. I thought that it can be a good idea to address
the market niche from the software side, and if implemented, perhaps propose it to hardware makers, who could push it to world’s markets if
they liked it. Quite a big challenge, but why not?
1.6.2
Sensor networks
While working with one of research teams lead by my thesis adviser, I
moved my study from the home-scale down to on-body network of sensors. Naturally, the sensor networks do not have to be biosensors wore on user’s body. They can also be part of home automation systems. But
there are some very interesting problems specific to Body Area Networks
(BAN). One of the problems I was careful to research on, was the need of real-time system responsiveness for certain applications. Another is the
power efficiency problem, which in case of BAN, is even more crucial than in home automation systems. While in the Smart Home inefficient
8
of BAN if a sensor stops responding too early, it may have bad outcome on a patient. And while in home devices can be powered from the grid, in
the BAN we would like to have an autonomic power supply (that we can wear and be mobile with it). I consider these challenges very interesting
and worth addressing, but for the purpose of my own system proposal, I eventually understood that I want to limit the target for it. I understood
that I want to develop a solution for the domains where RTT and low power profiling are not crucial. It does not mean that my design will
ignore power consumption optimization, contrarily – it will allow for implementing power saving profiles. However, nowhere in the
specification I will claim, that the system optimized for special conditions like in the case of BAN. Even though, I am sure that it can bring fair
amount of limited value to specialized domains of use.
1.6.3
High level development tools
For all the less then recent domains of various system integration
problems, there are plenty of tools and frameworks worth study. Throughout my study I found out that there are lots of tools that
significantly speed up the work on integration tasks. And it seems (from my own observation and experience) that the higher level the tools are,
the greater improvement in the speed of development. In fact, I realized that nowadays, we often address low level problems indirectly, via high
level simulations and emulations. For example, emulating ARM architecture under x86 or x64 and developing under it, may turn out to be
faster than actually connecting to the device and managing the cross-compiling and cross-debugging this way. But using Virtualization as
9
an example is not accidental. In fact, virtualization allowed me to see an interesting feature of high-level development. It allows for hiding things
from the user experience. Running web service from within a VM will make no difference to the network. On the top level it will be exactly
same as if the web service ran on an actual machine. Have you ever tried to run VM inside of a VM? This is an example of one high-level solution
that allows for implementing other solutions on yet higher level in relation to itself. And finally it hides the fact that the level has increased
from the user. I became particularly interested in this concept, as it allows for extending current designs by building around them, with the same
effect as if the changes were made internally. I wanted my system to have this feature. If there is a physical device connected to the network, I
would like to write a virtual one, that implements additional features, and then handle it over to the network just like if it was a physically upgraded
device. Actually, the WHOLE system follows that psychology. It combines all the physical and virtual devices into ONE large virtual
device. And if needed, it can then be added as a resource under another network (just like the VM inside of a VM).
2 Problem analysis
2.1 Problem definition
Although in current times we have access to a massive market for Customer Electronics and we have lot of standards of communication both wired and wireless, but it is not
trivial for an average user to integrate his devices and programs into a manageable collective population of software and hardware resources.
10
Let’s think of a bare bone PC for a moment. Because it is equipped with standard I\O ports, lot of additional hardware can be attached to it, increasing its functionality.
Connect a printer, and PC can print on paper. Connect a scanner, and PC can digitalize paper documents, even use OCR software to turn it indo editable digital text. Add WiFi
network card and the PC can surf the Internet and connect to even more hardware peripherals wirelessly. It can now control your DSLR if it has WiFi support too. Add an
IP camera and your PC can carry out surveillance in the house. It can communicate with a Smartphone. Connect an Ethernet-controlled power outlet, and you can power on and
off just any electrical device around the house. And since the power outlet is connected to the PC, and the PC is connected to the Internet, and so is a Smartphone, an electrical
device in the house can be turned on and off remotely from any location on Earth provided only that there is Internet connection available.
The way it has just been presented makes it seem that it is quite easy to carry out integration of devices into highly custom systems. This thesis explores the integration
problem and later on introduces an alternative way address this problem. Consider a scenario:
A fictitious user named Bill owns home audio system, PC, DSLR, and couple of mobile phones. Bill is not an engineer, but he’s a computer hobbyist. He would like to make his
home more intelligent and interactive. He wants that when comes back home, the home can recognize him and play music based on his preferences. However, if Bill’s wife
comes home first, the home would play music based on her preferences instead. And besides, every time there is a friend visiting, while Bill’s not home, the home would try
to recognize friend’s face and look it up the local database. Voice message would greet the guest. Also, the home would email Bill with the photo of visiting friend and the time
11
using components that are aleary in his house, but were so far being used individually. He wants his DSLR to hang on the wall in front of the entrance and take photos in a time
loop, then send them wirelessly to the PC. PC would run a face recognition software to detect people’s faces on these photos (if any). If a face is detected the software would
look it up in a database. If it matches Bill or Bill’s wife, PC would play music from assigned playlist on the home audio system. Otherwise, if it is someone else, while Bill’s
out of home, an email with the person’s photo would be sent to Bill. So if Bill’s mom came by to feed Bill’s cat while Bill’s out of town, he would get notified by email. Or if
a burglar has entered his house, Bill would have his photo to show to the Police.
In the scenario above Bill has components to address particular tasks within the scope of
the overall problem individually. However, he needs to implement the integration to turn these “indivituals” into “team players”. Turns out, that this is not a trivial problem to
solve due to few key factors:
Selfish hardware hardware that is designed to be able to connect to a network is not designed to serve as a resource to the network (a DSLR may have WiFi connectivity built in, but its design limits the way it can be used once connected to only a dedicated domain,)
Selfish software Commercial software owned by the user is not designed to allow the use of its functionality outside of the dedicated domain either (a photo browser can have face recognition functionality, but it does not provide this functionality to other applications)
Middleware challenge The difficulty level of implementing a middleware to enable software and hardware components to be used as shared resources arises as each component requires custom approach to building a standardized
12
wrapper/handler/driver around it with custom communication protocol (be it IPC, Serial port or UDP or TCP/IP signaling, web API or other)
RPC challenge A custom RPC-enabling layer would need to be developed prior to being able to implement high level code
The above table will serve as a break-down of the problem definition, and will get
referenced to later on.
2.2 Current designs and solutions potentially useful to
address the problem
Various protocols and standards have emerged over the past several decades to address
the challenges of distributed computing. From EDI (Electronic Data Interchange) to SOAP (Simple Object Access Protocol), to Web Services with WSDL, there are quite
some of standards allowing for integration and interoperability of various systems. These are on protocol design level and software level. On the other hand, for an inter-device
integration the physical architecture also has to allow for communication to be carried out. In wired communication we have USB and Ethernet cable support as two very
mainstream options for interconnecting various electronic devices. For the Wireless we
have Wi-Fi and Bluetooth as two other globally popular standards.
Naturally, these are not the only ones, but I chose to focus on these as most common
ones for connecting peripherals (based on my own observation on CE market and computer accessories/peripherals market). Pointing out these few standards will become
especially relevant when discussing integration in the mainstream market, where end users care for easiness of connecting their devices. In the industrial environment, or in
very specialized cases, some other standards might be chosen for matching certain needs more adequately.
13
There are several existing ways to address the factors of “selfish hardware” and “selfish software”. In practice, addressing hardware customization is in deed same as addressing
software customization, because what we want to actually customize is the firmware within a device. Otherwise, we can customize the device driver, in which case
customization takes place outside of the device. Yet higher level of customization is in the layer of the user application controlling the hardware through its driver, in which
case the distinction of hardware and software customizations is no longer needed. From the user’s perspective there is no hardware versus software concern. What user cares
about is the customization of the way a resource can be used. Addressing this part of the problem is possible through a range of existing technologies: from plugin-based
development (ie. control-over-http plugins), to a much higher level M2M protocols, to solutions compliant with the Internet of Things. The IOT however should be considered
a still-emerging solution rather than an existing one.
The middleware challenge (Ch. 2.1) has to be addressed by implementing a
wrapper-code around existing drivers and/or user applications. During the research there were no 3rd party solutions found that can automate this process for a typical binary
executables, but few wrapper-building frameworks were found for dedicated use:
1. Launch4j (http://launch4j.sourceforge.net) is a cross-platform tool wrapping Java
applications distributed as jars in Windows native executables. The tool allows an automated building of wrappers because the JAR applications incorporate meta data
describing entire headers of functions compiled into an application. In case of OS native binary executables, such meta data is not present, or is not complete. Shared
libraries have meta data describing names of exported functions, that can be extracted, but not the list of the arguments. This has brought a conclusion, that there might not
14
the source-code stage. On the source-code level, there are many libraries that can add RPC support to an application, like XML-RPC or JSON-RPC to name just few
popular. It will get examined later on, how much work from a programmer is needed to add RPC to an existing application. It will also be examined if the proposed
alternative solution can actually turn out simpler.
2. W4F is an approach to build web applications based on middleware architecture with
Web wrappers, also called adapters (Sahuguet et al, 2001). This case is worth mentioning because it shows that not only a de-facto application can wrapped. W4F
communicates with data sources published in the Internet, not applications. This brought the final conclusion: anything that contains enough of meta description of
itself, can be integrated with a middleware layers relatively easily and allowing automation in the process – which is not the case with OS native binary executables.
When it goes to the RPC challenge (Ch. 2.2), it is passed the concern for how to integrate particular RPC standard with existing application. In this stage the concern is actually
deploying the RPC environment. In a generic scenario for RPC standards mentioned before, there is client-server environment to be built. Chapter 3 will propose some
alternative in details of such architecture.
A new solution for integration and interoperability of SW and HW resources would have
to somehow bring improvement as an alternative to use technologies mentioned above. Otherwise it would need to allow for incorporating these technologies while still
introducing significant improvement to the overall problem.
2.3 Defining targets for potential solution
2.3.1
Value to users
End users are the principal target because the main focus of this research
15
computers and applications running on them. This target group will naturally care for an integration solution to be inexpensive, easy to deploy
and use, with user-friendly interface, and to be very popular, so that there is a big community working on it. This target group will also care less for
some very advanced and low-level aspects of the architecture used for realizing the integration, as long as the solutions brings them positive user
experience. One exception is the energy consumption efficiency. It is where users will likely care to a greater detail. Users can find it generally
attractive to have their homes delivering the Sci-Fi feel of being able to control all the devices as a collective network that they can program to
reach high level goals. That kind of experience would follow the concept of a smart home, which is a growing trend:
“The global home automation systems market has been forecast to
increase at a compound annual growth rate (CAGR) of 35.5% through to 2015, having already boomed from 237,000 systems shipped in 2007, reach more than 4 million by 2013” (King, 2012).
And the home automations, apart of the user experience aspect, is the key to having effective energy management in the house:
“Systems from Savant, Crestron, Vantage and others can even be
programmed to turn off or turn down devices if preset energy use ceilings are reached—though this will likely require programming costs. Even lower-end systems being sold by the likes of ADT, Comcast, Verizon, Vivint and Alarm.com can be programmed with basic macros and timed events to turn things off when someone leaves the house and the security system is armed.” (Castle, 2011).
16
The proposed solution can address the power efficiency concern in at least two ways:
1. its own architecture can follow a design optimized for minimalizing power usage,
2. it can provide an interface for 3rd party systems to deploy power-saving policies by issuing commands to power devices on/off
or to put them in (or wake from) the sleep mode.
2.3.2
Value to businesses
The value to the business is dual: one set of benefits is related to the
internal use (powering the micro environment = cost) and another relates to addidional business opportunity from having end users as primary
target for the solution (powering the macro environment = revenue). Firstly, the solution may help the business to run its operations in more
elastic, perhaps easier, less expensive way as compared to hiring System Integrators for implementing highly customized solutions. Saving costs is
also earning.
Secondly, any popular standard or feature is potentially a direction for a
product or service providers to create an added-value to end users as there will be demand for it. Ideally the solution should give businesses a
competitive edge in either introducing innovative features to their products, or just simply allowing them to implement more products and
services (or same number but faster or cheaper) than without the solution. And with more compatible/compliant products on the market the more
17
2.3.3
Value to industry
The proposed solution is not especially targeted at industrial use but is dedicated for spreading through manufacturing industry.
When looking at how brands are selling their CE products it is easy to see that the hardware factor is the least crucial in determining which brand
will take a market share away off another. Essentially, vast majority of branded products is being built by OEM manufacturers and in terms of specs the whole world’s market has access to pretty much same resources,
with only difference in costs, as every buyer gets his own deal with the
supplier. What end users see as differences in specs, is actually a result of strategic product segmentation, and has little to do with technical
advancement of the brand itself. Having essentially the whole market getting same hardware technology from same primary sources (major
OEMs), what gives their products competitive edge is largely the design and features. The more interesting the design and the more feature-packed
the product is, the bigger chance it stands to capture end-user’s attention. And there is an aspect here worth more study: Customers recognize
brands and pay high margins for big names, while OEM manufacturers who remain largely unknown to end-users, and who sell products at low
costs minimally above the cost of production (but in large volumes) are the ones who makes the most money at the end (like when realizing that
Apple considered by end-users as a brand making lots of money falls short compared to the money that their supplier Foxconn make*). How is
that relevant to the research covered here? A solution proposed later on in the chapter 3 can potentially have a significant influence on the
18
Smartphone, the brand, the design, the features are all important. A device like Smartphone is a self-contained product, and via implemented wired
(USB) and wireless (Wi-Fi/BT) standards, can communicate with lots of other devices. But built-in specs are not designed to be
exchangeable/upgradeable like in PC computers. When it comes to DIY approach, and end-users interested in adding features to their intelligent
and automated environments, things become more interesting. A cheap, “brandless”, simple sensor, that would be no value to an end-user if it was
not self-contained and able to be used stand-alone, becomes a tasty treat if the same user can just bring it home and add to an existing network of
resources to enable new capabilities to his automated home. Actually, one of more important reasons why end users would not buy very cheap and
common electronic components, sensors etc. and would instead buy more expensive, more complex, self-contained, branded products, is that they
would not know how to make use of these cheap components anyways. It’s simply easier to buy the later, even if one ends up with a product
packed with sensors and other built-in components, only to find out that lot of them will be at no use to him. This can change dramatically if there
is a common integration solution, that would allow anyone to buy single, inexpensive components, bring them home, connect and just use. It would
change a lot in the market, if in order to have our homes to intelligently response to environmental conditions we would only need to bring a box
of cheap sensors from an electronics shop and have it all being used in conjunction with the PC, laptop or a Smartphone that is already there and
19
distributed device. Currently, although far more expensive, it is just easier to either buy a dedicated system including a hardware + software bundles
and have it deployed in our house, or to hire a System Integrator.
In the same time, it would not necessarily mean that OEM manufacturers
would start to go directly to end-users causing brands to loose business. It would give businesses more opportunities to sell more, with less effort, at
lower prices and with lower cost. The reasoning behind it is, to put it the simplest way, is that most of end-users are lazy. Even if an end-user can
easily DIY his own automated home, he will probably (author’s point of view) not be very interested in developing all the applications to be run in
his integrated system by himself. This is where business can start making more business. Buy cheap components, spend no time on external design,
and branding, but implement features, and write applications that will be able to co-use these features with other features that will be available at
the run-time when the product becomes a member of a larger distributed system. There are at least 2 possible changes that can happen in the
manufacturing industry if the proposed solution went viral:
1. End users would expect devices to be increasingly compliant with the
new standard, so that after purchasing goods there is more freedom to create applications for these goods even outside of the domains they
(devices) were dedicated to serve originally
2. Manufacturers would be able to start targeting mainstream market
with more component-like products, which have little market value as stand-alones.
20
2.3.4
Value to developers
Not only hardware makers can potentially find new ways to design and sell products, but also software developers can be given similar
opportunity. Looking at any end-user level software there is a large number of functions implemented in it, but in general these functions are
only being used to solve problems that the program containing it was dedicated to solve. What if an end-user figured out another way to use
certain features of a complex program to solve a problem that was not expected by the program writers to be solved with it, or was not thought
to solve the problem in the same way? Unfortunately for the user, there might not be a way to extract a subset of functions from a program and
call them from outside of that program (unless the software was originally meant to be used in this way). Perhaps allowing the user to be able to
selectively call functions of a program that he finds useful could make a great added-value to the software product. It could also give software
sellers more possibilities of implementing sales. For example there could be a price structure for a software based on a license that either allows the
user to selectively call its sub-functions remotely or not. Perhaps it could give more flexibility to sell SaaS (Software as a Service) where user can
selectively call functions based on his subscription. Google has released a freeware software called Picasa, that allows to manage, browse, edit and
share photos. Among many features contained by it, there is one that allows for face detection and recognition. It is a great functionality, but
Picasa only uses it to tag people on photos and allow search in the library of photos by presence of people tagged on them. If that functionality
21
analyzed and if a face was detected and recognized some further action could be taken based on the identity of a person or persons (and that
further action could call functions from other programs or other devices). Picasa is only one of many software products offering face recognition
and tagging functionality. Very similar features will be found in Adobe Photoshop or Corel Paintshop Pro. There is even a chance that the
software maker has not implemented its own computer vision library, but that he licensed a third-party library. It is even possible that same library
is being licensed by many software makers. Perhaps that original library could be added as a resource to an integrated system, so not only software
makers, but anyone could use it (be it for free or by purchasing a license). And then Google, Adobe and Corel could release their software without
the built-in face recognition functions, but marketing their products as compliant to a general standard, so that they can integrate with computer
vision library if only one is present in the system where the software is running. Just like a Mobile Phone Carrier can buy a Smartphone from a
vendor, put a logo on it, create a custom ROM with Skype application installed, and sell it advertised as a phone with Internet Call support. Even
more interestingly, instead of having many programs running on a machine having their own “copy” of the vision library loaded into
memory, there could be only one instance of the vision library and all the programs on all the devices connected to the system could use it as a
service, or as a shared feature.
The integration system itself would be a new type of platform for
22
distributed across devices and programs. Where any feature can be either software or hardware implemented, while it remains transparent to the
user. Where applications could be designed to use resources before knowing what these resources might be during the run-time. And possibly
in a programming languages that handles RPC transparently so an RPC looks like a regular local call from the source code level. This will get
further discussed in chapter 3.
2.4 Designing benchmarking criteria
2.4.1
Objectives
The principal objective for proposed solution (see Ch.3) is to improve (or
replace) alternative/competitive solutions on the market. Even if the competing systems incorporate more numerous and/or more advanced
features, there are still other factors that might potentially help in reaching better reception from the market: simplicity to end-users, simplicity to
developers and price effectiveness factor. The more complex (feature rich) a product is, the more difficult it is to keep a simplistic, intuitive interface
for end users. There is always a chance, that the user when comparing products, can choose the one with a smaller feature set because of a
simpler, more intuitive interface and a better user-experience. Since the evaluation of these objectives will not be possible without actually
releasing the solution to the market and waiting for market’s feedback, a benchmarking model will be designed to approximate system’s potential
23
2.4.2
Factor 1: Simplicity to End-users
This factor is related to how well the proposed solution can satisfy End-users liking in aspects such as:
- Intuitive and simple User Interface (UI)
- Low learning curve needed before able to deploy and use
- Seamless scalability: growing the system require the little work and special attention
- Richness of applications: less advanced system stands chance to become more popular against a much more advanced competitor if it
exposes significantly more use applications for the end-users (becomes useful for more things)
2.4.3
Factor 2: Simplicity to developers
This factor is related to how easy it is for developers to build more solutions on top of the provided system, or to integrate it with other
systems. The easier it is, the more development projects are likely to grow around the proposed system, and use applications go to End-users
(helping the first factor).
Another aspect is that once developing under the proposed system
becomes simple and innexpensive, it can help more End-users to also become developers. This can grow the rate of End-user development
(EUD) for the system. Computer users nowadays stand for a huge number and diversity (Scaffidi et al 2005), spreading across many disciplines and
including professionals in many fields: managers, accountants, scientists, engineers, teachers, health care workers, administrative workers, even
24
change on any basis from annually to even daily. If so, then also their needs and expectations for any software used vary frequently, and become
more complex. Professional software makers fall short in meeting all these needs directly due to their limited domain knowledge or/and
because of too slow development process (Burnett and Scaffidi 2011). This shows how increasingly important is the scale and role of EUD. One
of the definitions of EUD states that it is “a set of methods, techniques and tools that allow users of software systems, who are acting as
non-professional software developers, at some point to create, modify, or extend a software artifact” (Lieberman et al 2006). End-users can
particularly design and/or customize the UI and functionality of a system, when they need to better fit their specific context and needs. Also,
because end users outnumber professional software developers by a factor 30:1 (Burnett and Scaffidi 2011), EUD enables much more people to
participate in development. In order to encourage more EUD happening, some specifically appropriate tools and features are significantly
important, that can make development processes highly usable and quickly learned. It is because unlike professional developers focused on
durable software assets producing continuous revenue streams, the end users rather do programming for reaching short or medium-term goals –
sometimes they just want to write code “on the spot” to use it instantly, and without carrying if they will later need to ever use it again. Enabling
more EUD over the system requires a proper, dedicated development framework. Because the discussed system is dedicated to enable
25
forming a network, it shows lot of similarity to End-user programming of Smart Homes. This is a very challenging issue. A great way to realize
scale of complexity was to propose a model for networked artifacts inspired from molecular chemistry (Coutaz, 2008). To provide end-users with the capacity to “program” their interactive spaces like their home, a
way to handle complexity of networked artifacts must be found. There is
an analogy to this complexity found in chemistry where a smart artifact is modeled as a composition of physical and digital atoms whose
configuration evolves under particular conditions. Software services are composed of digital atoms only, whereas conventional objects of the real
world are strictly built from physical atoms. A smart house is considered as an interactive space represented by a unique macro-molecule of both
digital and physical atoms, or a set of smaller molecules. Then just as in Chemistry, the nature of the events that trigger a reaction has an impact on
the resulting product. This way to look at the EUD problem would rather discourage average end-users to participate in the development, which is
why hiding all the complexity from the user while still allowing him to produce results, becomes crucial.
2.4.4
Factor 3: Price effectiveness
New system should allow for an inexpensive integration. An average user should be able to afford using it. This factor is much more reflecting the
business aspect then just focusing on providing technology solution. While the actual marketing planning is not within the scope of this paper,
it is relevant to some extent when it comes to architectural design. Regardless of the form in which the sales of the solution might be
26
implemented in the future, in the architecture level it should be considered how to accommodate as many possibilities as there might be. From the
Chapter 1 we have seen some new trends in sales design surfacing, like selling software or infrastructures as a service for example. Even though
this paper only acknowledges the business aspect to become relevant at some point in time beyond the scope covered, but keeping it in mind can
greatly help in designing system architecture.
2.4.5
Justifying the selection of factors
To justify the 3 factors given a case study of Android’s (operating system)
market share was arbitrary chosen. Below are the worldwide smartphone unit sales and market share in the 1st quarter of 2012, by IDC
(FRAMINGHAM, Mass. May 24):
Mobile OS 1Q12 Unit Shipments 1Q12 Market Share
Android 89.9 mln 59.0% iOS 35.1 mln 23.0% Symbian 10.4 mln 6.8% BlackBerry OS 9.7 mln 6.4% Linux 3.5 mln 2.3% Windows Phone 7/ Windows Mobile 3.3 mln 2.2% Other 0.4 mln 0.3%
Furthermore, from the research by IDC we can see growth of the above market shares compared to Q1 2011:
27
iOS (iPhone) – 88.7% growth Symbian – 60.6% decline
BlackBerry – 29.7% decline Linux – 9.4% growth
Windows Phone/Mobile – 26.9% growth
Particularly interesting is the growth of Android and the second biggest
player on the market, the iOS. How an OS can reach such success in a crowded market? Firstly, let’s focus away from the fact, that part of Android’s success has come from Samsung, which accounted for 45% of
all Android sales in the quarter. Still, Android is the latest into the market
from all the top popular OSes in the Mobile market:
“The popularity of Android and iOS stems from a combination of factors
that the competition has struggled to keep up with, (...) Neither Android nor iOS were the first to market with some of these features, but the way
they made the smartphone experience intuitive and seamless has quickly earned a massive following” (Ramon Llamas, Research Analyst with
IDC).
That illustrates how a newly proposed solution can win the popularity
among end-users if only focuses more than the competition on user’s experience. Another crucial aspect is the number of available applications,
in which case, using the mobile OS market as a case study brings in even more interesting observations. From the top level, Android and iOS both
are leaders in the number of apps available to their users. This measurement is greatly influencing the end-users when it comes to
28
an operating system, or in broader context, a Smartphone powered by a particular OS. This case study will make few key observations in process,
that will reflect in the thinking on the solution proposed in Chapter 3. To start with, let’s point out, that in the case of mobile OS market, the
software becomes a great deal when it comes to the value of a hardware. One could speculate if a Smartphone great in hardware design, could
represent a high value to end users if it was running on an OS with very low number of applications. This question also reflects consideration for
the solution proposed in this paper, that is also implemented in software. But there is more to learn from the example of Android and iOS study in
the area of apps. Great observations has been made by Darcy Travlos in “Five Reasons Why Google Android versus Apple iOS Market Share
Numbers Don't Matter” for Forbes. She has made some research on Gartner’s Q2 2012 mobile sales unit report, and some of her findings are
as follows:
- Apple does money on hardware, Google does not
- Google gives away its Android operating system and earns on Search
and delivering ads
- Apple makes money on every iPhone and iPad it sells, even before an
ad is delivered to the device
- Apps are more popular than mobile web, and Apple is winning this race by any measure
- on mobile devices, 94 minutes per day are spent on apps compared to
72 minutes on the web
29
- people are spending for Apple’s apps – Apple’s apps make money
- App store generates $5.4M/day for the 200 top-grossing apps while
Google generates just $679K for their top-200 grossing apps. That is almost a 8:1 revenue ratio
- More of Apple’s apps generate revenue, while most of Google apps are free (67% of apps on Apple are paid for versus 34% on Google)
- Apple apps make developers money ( Android developers made
$210M in all of 2011, compared to the $700M pocketed by Apple iOS developers in the Q4 2011)
- getting paid attracts more developers to Apple
- ppStoreHQ estimates there are over 43K Apple iOS developers and 10K Android developers (Travlos points that it is because iOS
developers earn more)
- By estimate, for the very same app, a developer will earn $1.00 on the
Apple iOS version compared to $0.24 for the Google Android version After looking at the findings listed above, a very important consideration
for the research comes to mind. Even if the focus of this very paper is on proposing a technology-based solution, not a business plan, the aspect of
a potential revenue model for the solution if delivered to market, seems clearly crutial to satisfy objectives presented earlier. The 3rd factor
(Simplicity to develepers) in particular requires the system architecture to consider later potential earning opportunities for developers. Before
concluding this portion, it is further helpful to consider 3 reasons for why many new popular apps are developed for Apple iOS first, given by
30
- Apple has fewer form factors (3 iPhones, 3 iPads) compared to thousands of Android devices
- Apple’s app approval process requires developers to guarantee a
certain quality
- Apple user demographic is more affluent, an earlier adopter and more
loyal than other brands
What constructive does the fact that iOS developers have fewer sets of hardware and middleware issues to address than do Android developers
bring to the research covered in this paper? A suggested architecture will be targeted to numerous and diverse base of hardware platforms. In order
to achieve Apple’s advantage, the developers under this architecture should be given a very generic development environment, that can hide as
many form-factor-specific details as possible. At least in greater deal than the competitors have managed to. Another Apple’s advantage is in
building credibility in guarantying quality apps (apps approval process). In this paper the research does not stretch further into the app market
business model, nor does the proposed architecture include it, but it should propose alternative ways to achieve credibility among users
(considering specifically the use applications). As for Apple’s third advantage, the scope of this paper does not reach to the problem of user’s
loyalty to a brand, but some particular demographics can be especially targeted, to reach as many of the early adapters as possible once the
system gets released.
The third factor treats about the appealing pricing strategy. To create an
31
integration with various possible implementations of sales strategies. In particular an effort should be made to design the system in the way to
easily enable some of the trending sales approaches.
2.4.6
Benchmark summary
To allow comparison with other existing solutions, a benchmarking model will consist of two units: User Experience Unit, and SDK Unit:
User Experience Unit for benchmarking
Intuitive UI Does the solution offer a dedicated Graphical Interface for end-users that is easy and intuitive?
Learning Curve Is the learning curve reasonably low for end-user to deploy and use the system?
Scalability Is it made easy for end-user to increase number of resources in the system?
Applications base Is there a public base of applications for end-users to use?
Affordability Is it expensive for an end-user to get the system, deploy it and use?
SDK Unit for benchmarking
Package What does the SDK package consist of?
Learning Curve How much new knowledge a developer needs to acquire before being able to develop for the system?
Portability Is it easy to port existing project to this new framework?
EUD support Is there a natively built-in development kit for End User Programming?
3 Solution proposal
3.1 Overview
Designed is a middleware system for allowing the integration of arbitrary electronic
devices and software into a collective population of resources capable of cooperation. The complete architecture consists of 5 layers, that are covered in
