With the advance of the science technology, the Grid has been gradually developed. Grid systems include the network infrastructure and the software architecture, which provide distributed computing resource platforms. Because of the development of internet and the technology of computing, the internet users not only share files easily, but also share the extensive computing resources. Grid systems provide the network services, communication channels and computing resources. The main functions of the Grids are using distributed computing resources more efficiently and providing users transparent services in Virtual Organizations (VOs). These computing resources may be scattered in the different organizations or the different regions. At present, Grid applications have adopted in various fields such as the medicine simulation, medical and high-energy physics.
The Grid systems could be broadly characterized as the computational Grid and the data Grid. Computational Grids focus on computing resources sharing, and data Grids focus on storage resource sharing [43]. IBM developed e Business on Demand (eBoD) [12] in 2002, and arise the On-demand Grid development. In the future vision of eBoD, the user of enterprises use computing resources would like to use as electricity. The user could start the service when need it, and can stop it when the user does not need service anymore. The eBoD made a big change from focusing on storage capacity scalability in the past to choosing the computing resources according the user’s demand. In the Grid system frameworks, satisfy the real time demand by uses or exchange the computing resources all over the world, such a concept known as computation on demand. Before the Grid technology been developed, Distributed.Net (Dnet) [9], Search for Extraterrestrial Intelligence (SETI) [41] made the
computing more efficiently by use distributed resource, and laid the foundation of Grid science development.
Overall, the most important concepts of the Grid systems are capability of “the dynamic computing resources” and “cross multiple virtual organizations”. The term “Grid” was coined in the mid-1990s to denote a proposed distributed computing infrastructure for advance science and engineering [18]. The concept of power Grids means that when user use electricity not necessary understanding where the power comes from, let the user can get the cheap and reliable power [18]. The first persons complete definition the Grid are Dr. Ian Foster, Dr. Carl Kesselamn and Dr. Tuecke. The father of Grid computing Dr. Ian Foster said the view of Grid computing is “Grid computing solves the issues of resources sharing and cross multiple virtual organizations dynamically, and can be flexible, safe and coordinated to achieve resource sharing＂. July 2002, Dr. Ian Foster pointed out three characteristics of the Grid computing more concrete [15], listed as the following:
(1) A GRID coordinates resources that are not subject to centralized control.
(2) Uses standard, open, general-purpose protocols and interfaces.
(3) To deliver nontrivial qualities of service.
On the other hand, Peer-to-Peer computing, such as Napster [36]、Gnutella [21] and Freenet [19], these file-sharing systems are similar with the Grid computing system. They both share resource from community organizations. With the Grid systems scale expand, have begin need to solve some issues, such as self-configuration, fault tolerance, system scalability, and other topics [16]. In many Peer-to-Peer studies provide above solutions. Peer-to-Peer systems focus on dealing the issues with instability and instantaneous flow, self-configuration and fault-tolerant. However, the developments of Peer-to-Peer technology mainly work in vertical integration and develop application, rather than work in constitute the universal protocol and the standardization framework. In contrast, the Grid computing devotes to
develop the open standard, and ensure that Grid systems can be integrated with the applications by provide commonality interface. Of course, there will have new technologies or tools be produced in the process of the Grid systems development in the future. Represent that the standards also need to check and amend at anytime.
When Peer-to-Peer technologies among the delicate and complex applications, such as decentralized structure, desktop application and network computing. Begin to look forward the Peer-to-Peer and the Grid computing could make a strong integration, and it will be a new beginning and output in the future.
In this study, we present a light-weighted Grid middleware for computing resources collection, integration and use. We expect that we can help job execution more efficiently and improve the idle computing resource usage by this Grid middleware.
1.2 Problems
Currently, the biggest challenges in the Grid systems are how to establish and manage sharing relationships from different VOs. These challenges must be solved by the new technologies.
Some of the Grid system frameworks in job submit need to specify VO to execute, and allocate the computing resources by using the middleware (such as condor) to allocate resource in the same VO. A credible Grid middleware should provide convenient and flexible environment for the Grid user, but in different Grid systems also have various types of middleware, the accompanying problem is difficult to integrate computing resources.
Most of the Grid systems do not have same communication channel (pipe line) between the VOs, coupled with the consideration of security and the target of different VOs, result in difficult come to integrate computing resources. If we adopt Peer-to-Peer technology to achieve computing resources sharing, management and access control from the different VOs.
The resource adjustment mechanism in the Grid systems can be divided into global and local adjustment mechanism. Most of the global adjustment mechanisms are deployed by resource broke or agent. Resource broker play the computing resource allocator role between the user and the resource provider in the Grid systems, help users find suitable machine for their job and complete the information access transaction. Currently, the Globus Toolkit does not provide resource broker function, and must be compatible with other resource allocation systems to allocate computing resources, such as EGEE (Enabling Grids for E-Science and Industry in Europe) [64] developed the middleware named gLite, and it adjusts the job by the global resource broker. Global resource broker could assign the job to the appropriate computer elements (CE) to execute. If the jobs wait for execution in the job queue of the CE, the global resource broker unable to re-schedule these jobs anymore. It illustrates that dynamic of resource broker inadequate.
In the fact, computing resource is extremely dynamic in the Grid systems. For examples, the computing resource join/leave at any time, different computing power and the network bandwidth speed. All these factors will influence the performance of job execution. In this heterogeneous environment, computing resources may in a high load or light load conditions at any time. Some of specific computing nodes have high utilization , such as the computing nodes who own a better computing power, the user will submit the job to these nodes first.
Many of the computer nodes have proof computing power, they often be ignored. If we can improve the usage of computing resource and reduce the idle chance of computing resource, the efficiency of the job execution will be able to improve. Many load balancing and load sharing studies solved the computing resource adjustment and resource imbalance by job migration mechanism. In traditional practices, considered less on the significant heterogeneity of Grid systems or the experiment methods were not suitable for the heterogeneity
environment. Therefore, how to take the right measure standard is the important issue in load balancing or load sharing studies.
1.3 Purposes
Grid systems not only provide computing resources, but also make computing resources do the maximum validity of the utilization. The principal thrusts in this study are computing resources collection, integration and use, and with three main contributions to (1) integrate computing resources by Peer-to-Peer technology (decentralized structure) (2) build the light-weighted Grid middleware by Peer-to-Peer technology, and (3) to improve the utilization of idle computer resources by job migration mechanism, and to achieve the efficiency computing resources sharing. At present, the most of the Grid systems middleware are using Globus Toolkit and it as a major standard component. In this study, we develop the number of modules and supply a light-weighted Grid middleware system prototype by Peer-to-Peer technology, named Service Oriented Roaming System (SORS). According by construct the unite communication pipeline in SORS, we makes the message transfer and job execution more efficiency. The modules in SORS consist of the Basic Service, File Transfer, Information Service, Execution Management and Load Sharing. Through these modules support, achieve the integration of resource information, the utility of computing resource and the load sharing. In load sharing management, we use the distributed strategy, and consider the heterogeneity factors of the Grid system, including the bandwidth speed, the ability of job execution, and so on. Let the job choosing the most appropriate computing resources to execute job, and reducing the idle chance of computing resource.
1.4 Restrictions
The restrictions of this study, Grid systems are deployed by different regions, virtual organizations, storage systems, software architectures, and with a high degree of changing. In our experimental, we consider the budget and the management of the facilities. Our experimental environment adopts the Grid organizations that the laboratory current joins (Taiwan UniGrid and Tiger Grid), and come to job migration into cross site and cross multiple virtual organizations. The scale size of the experimental environment in this study is small than the actual environment, but can help the managers to control computing nodes state at any time. The changing and different of experiment results also small than the large scale Grid systems.
Security is an important consideration in the Globus Toolkit. Different virtual organizations require different certifications to ensure security when the members use computing resources. Because of this study focus on cross sites and cross multiple virtual organizations by Peer-to-Peer technology, in order to complete control all the computing nodes. Therefore, we do not consider the Grid Security Infrastructure (GSI) of Globus Toolkit in this study.
1.5 Notations
The related terms and the meanings used in this thesis are given in Table 1-1. These symbols will be used in the following chapters.
Table 1-1 Te
rms and their meanings
Term of name Meaning
L_AL
Local Average LoadR_AL
Remote Average LoadL_LB
Local Load BarrierR_LB
Remote Load BarrierJob_idle
The idle job in the job queueJob_Size
The size of a jobAVG_Bandwidth
The average bandwidth speedMigration Cost
The job migration costLocal_JRT
The job response time in local siteRemote_JRT
The job response time in remote siteAVG_JRT
The job average response time of all jobsLocal_ idle_ length
The idle job length in the local site job queueLocal _running _ length
The running job length in the local site job queueRemote_idle _length
The idle job length in the remote site job queueRemote_running_length
The running job length in the remote site job queueLocal_finish_time
The job finish time in the local siteRemote_finish_time
The job finish time in the remote siteThe Grid system includes the network infrastructure and the software architecture, to provide distributed computing resources platform. Because of the development of internet and the technology of cheaper computing, the internet users not only share files easily, but also share the extensive computing resources. The most important concepts of the grid systems are capability of “the dynamic computing resources” and “cross multiple virtual organizations”.
Currently, the biggest challenges in the Grid system which are does not have same
communication channel (pipe line) between the VOs, coupled with the security consideration and the different VO target, result in difficult come to integrate computing resources. We adopt Peer-to-Peer technology to achieve decentralized computing resources sharing, integrate by management and access computing resources from different VOs. Thus, we can improve the rational use of computing resources and reduce the computing resource idle chance, and the efficiency of the job execution performance will be able to improve.
The principal thrusts of this study are computing resources collection, integration and utilization. The three main contributions to (1) integrate computing resources by Peer-to-Peer technology (2) build the light-weighted grid middleware by Peer-to-Peer technology, and (3) to improve the utilization of idle computer resources by job migration mechanism, and to achieve the computing resources sharing. Let the job choose the most appropriate computing resources, and reduce the idle chance of computing resources.
We introduce the gird and peer-to-peer system, problem and propose in chapter 1.
Chapter 2 will discuss the related works, which include for grid computing, peer-to-peer technology, JXTA and migration technology. Chapter 3 will further discuss the system design, which include for development tool, system framework overview and the proposed algorithm.
Chapter 4 will describe the experimental environment in this study. Chapter 5 will explain the results and statistic obtained from the experiments. The last chapter will include related suggestions, directions for future development, and conclusions for the summarization of this study.