Coordination of Multiple Agents for Production Scheduling

全文

(1)Annals of Operations Research 75(1997)235 – 289. 235. Coordination of multiple agents for production management* Jyi-Shane Liu a and Katia P. Sycara b a. Department of Computer Science, National Chengchi University, Taipei, Taiwan E-mail: [email protected]. b. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA E-mail: [email protected]. The new challenges of “Agile Manufacturing” and distributed decision making entailed by decentralized organizations led to our interest in the study of computational cooperative problem solving models and coordination techniques for distributed production management. The goal of our research is to address the technical need of distributed production management and develop appropriate computational approaches to support adaptive, cost-effective responsiveness. In particular, we focus on the challenging problem of job shop scheduling, which has been one of the primary foci of production scheduling research. This paper presents a multi-agent problem solving model and an effective coordination technique for job shop scheduling. The model involves a group of agents; each agent is associated with either a job or a resource. A solution to a production scheduling problem is the result of coordinated conflict resolution in the iterative and asynchronous multi-agent decision making process. It is well known in distributed systems research that for tightly interacting, non-decomposable problems, such as job shop scheduling, the need for communicating partial solution results among parts of the system rapidly degrades system performance. On the other hand, limiting communication degrades solution quality. One can limit communication by employing shared memory, but this has the drawback that the shared memory becomes a bottleneck and, in addition, using shared memory limits decentralization. In our approach, we judiciously balance the above concerns. We limit interagent communication through a scheme that employs efficient, small and distributed shared memories, each of which is associated with and shared by a limited number of agents. We also exploit problem characteristics (e.g. disparity among subproblems) to design an effective coordination technique for the job shop scheduling problem. We have evaluated the utility of our approach through extensive experimentation on a variety of job shop constraint satisfaction and optimization problems with different optimization objectives. Our results show that our approach outperforms or gives comparative results with other state-of-the-art scheduling techniques on benchmark problems. Keywords: multi-agent problem solving, coordination, production scheduling. *. This research has been sponsored in part by ARPA Grants F30602-90-C-0119 and F33615-93-1-1330.. © J.C. Baltzer AG, Science Publishers.

(2) J.S. Liu, K.P. Sycara y Agile manufacturing. 236. 1.. Introduction. In the era of global economic markets, overseas competition has impelled the manufacturing industry to move toward the concept of “Agile Manufacturing” [17]. The capabilities to effectively and adaptively operate modern production facilities become increasingly important to the survival and success of business organizations. The need for new logistics and production management techniques to respond rapidly to changing market demands has spawned decentralized organizations and entailed distributed decision making for the common organization goals, e.g., reduce inventory, meet order due dates, reduce resource idleness, etc. Advances of communication services, both for human decision makers and computational devices, further enhance the plausibility of de-centralization. However, success of distributed management hinges on effective coordination, which has not been well studied and well understood in computational approaches. This new challenge leads to our interest in the study of computational cooperative problem solving models and coordination techniques for distributed production management. Distributed Artificial Intelligence (DAI) is concerned with the coherent problem solving behavior of a group of related individuals (computational agents) [12, 14]. As computer applications begin to permeate into our everyday activities, DAI research receives more and more interest. Recent DAI research has studied important issues, such as negotiation [40], interaction protocol [7], coordination [8, 18], when to communicate with other agents, what to communicate, and how to model other agents, etc. These studies have produced significant results and laid the foundation for real-world cooperative problem solving. Facilitated by the development of networked computer systems and practical concurrent processing architectures, realistic cooperative problem solving has been realized in increasingly complex domains, such as design [1, 2], flexible manufacturing systems [30, 32], office information systems [25, 29, 31], and communication network management [15, 39], etc. The goal of our research is to address the technical need of distributed production management and develop appropriate computational approaches to support adaptive, cost-effective responsiveness. In particular, our current study focuses on the challenging problem of job shop scheduling, which has been one of the primary foci of production scheduling research. Production scheduling has significant implications in manufacturing management. The effective scheduling of a facility enables faster response to customer demands, reduces its work-in-process, and increases its throughput and, therefore, is instrumental in achieving objectives such as gaining increased market share and increasing return on investment. This paper presents a multi-agent problem solving model and an effective coordination technique for job shop scheduling. The model involves a group of agents; each agent is associated with either a job or a resource. Lateral coordination among agents defines the process of distributed production management. A solution to a.

(3) J.S. Liu, K.P. Sycara y Agile manufacturing. 237. production scheduling problem is the result of coordinated conflict resolution in the iterative and asynchronous multi-agent decision making process. It is well known in distributed systems research that for tightly interacting, nondecomposable problems, such as job shop scheduling, the need for communicating partial solution results among parts of the system rapidly degrades system performance. On the other hand, limiting communication degrades solution quality. One can limit communication by employing shared memory, but this has the drawback that the shared memory becomes a bottleneck and, in addition, using shared memory limits decentralization. In our approach, we judiciously balance the above concerns. We limit inter-agent communication through a scheme that employs efficient, small and distributed shared memories, each of which is associated with and shared by a limited number of agents. We argue that design of appropriate coordination strategies should be based on characteristics of the problem structure. We examine one of the recurrent problem structures that involves task disparity among agents, e.g., some of the agents have dominant effects on the process of solution generation. We developed coordination techniques that take advantage of disparity among agents to conduct effective lateral coordinated negotiation. We use negotiation as a metaphor so that constraint conflict resolution can be viewed as a repeated negotiation process. This view has been expressed in other DAI conflict resolution situations [5]. The cooperative problem solving model and the coordination technique are applied to both job shop scheduling constraint satisfaction (with non-relaxable time windows) and constraint optimization (with objectives) problems. We conducted computational experiments on benchmark job shop scheduling problems and compared the results to other centralized scheduling techniques. The goal is to rigorously verify the effectiveness of our cooperative problem solving approach on predictive scheduling, which is an essential component of production management. We intend to build upon the results and further develop our cooperative problem solving approach to address other areas of production management, such as reactive scheduling, inventory control, job routing, resource allocation, etc. In the remainder of the paper, section 2 defines job shop scheduling (JSS) problems and describes a cooperative problem solving model for the problem. Section 3 presents the design of the coordination techniques for the JSS constraint satisfaction problem, which include the disparity structure of job shops, the coordination procedure, and the set of useful coordination information. Section 4 describes the conflict resolution heuristics of the agents. Section 5 gives an overview of the group negotiation process. In section 6, we extend the coordination techniques to the JSS constraint optimization problem. In sections 7 and 8, we report the experimental results of our approach on benchmark JSS constraint satisfaction and constraint optimization problems, respectively. In section 9, we discuss the evaluation of the experimental results. In section 10, we overview representative related work. Finally, in section 11, we conclude the paper..

(4) J.S. Liu, K.P. Sycara y Agile manufacturing. 238. 2.. Job shop scheduling. Production scheduling deals with organizing possibly related production operations over time and across limited resources. The diversity of human and machine activities are modeled in a wide range of scheduling problems of different conditions. We are concerned with a subset of scheduling problems, known as job shop, that has been one of the primary foci of production scheduling research. 2.1. Classic job shop model To present the classic job shop model, we introduce several basic notions. We adopt the definitions given in [6]. Operation:. An operation is an elementary task to be performed.. Processing time: The processing time of an operation is the amount of time required to process the operation; in most cases, setup times are independent of operation relations and are included in the processing times. Job:. A job is a set of operations that are interrelated by precedence constraints derived from technological restrictions.. Machine:. A machine is a piece of equipment, a device, or a facility capable of performing an operation.. Release time:. The release time (or release date) of a job is the time at which the job is released to the shop floor; it is the earliest time at which the first operation of the job can begin processing.. Due date:. The due date of a job is the time by which the last operation of the job should be completed.. Completion:. The completion time of a job is the time at which processing of the last operation of the job is completed.. Schedule:. A schedule is a specification of the execution of each operation on a particular machine at a specific time interval. A feasible schedule is a schedule that observes all problem constraints, e.g., job release time, operation precedence relations, and machine capacity.. Classic job shop represents a manufacturing production environment where a set of m jobs J = {J1,…, Jm} have to be performed on a set of n machines (or resources) R = {R1,…, Rn}. Each job Ji is composed of a set of sequential operations oprij , j = 1,…, mi, mi ≤ n, and can only be processed after its release date RDi . Each resource Rs has only unit capacity and can process only one operation at a time. Each operation oprij has a deterministic processing time pij and has been pre-assigned a unique resource which may process the operation. Since a job consists of a set of operations that has to be performed in sequential order, it is conceptually convenient to envision.

(5) J.S. Liu, K.P. Sycara y Agile manufacturing. 239. a job entering the shop, visiting different machines to have its correspondent operations performed, and then leaving the shop. A job’s routing is the sequential set of resources that the job visits before its completion. The arrival time of a job at a machine is the time at which the job leaves the previous machine in its routing, and is equivalent to the ready time of the operation to be performed on the machine. The model provides only a basic description of actual job shops and does not elaborate on more complex conditions. In order to focus the study on the effects of scheduling and to allow generalization of the experimental results, most research on job shop scheduling share the following assumptions to simplify the model. (1) A job has a fixed ordering of operations. (2) A job does not visit the same machine twice. (3) An operation may have at most one other operation immediately preceding or succeeding it. (No assembly operations.) (4) An operation of a job can be started only if its immediate preceding operation, if any, is completed. (No overlapped operations.) (5) An operation can be processed by only one machine. (No alternative resource.) (6) An operation can not be preempted during its processing. (7) Operations do not require explicit setup times. (8) The machines have unit capacities in the entire interval of scheduling (no machine down time). Job shop scheduling problems [10] involve synchronization of the completion of m jobs J on n resources R (machines). The problem (hard) constraints of job shop scheduling include (1) operation temporal precedence constraints, i.e., an operation must be finished before the next operation in the job can be started, (2) release date constraints, i.e., the first operation of a job can only begin after the release date of the job, and (3) resource capacity constraints, i.e., resources have only unit processing capacity. A solution of the job shop scheduling problem is a feasible schedule, which assigns a start time stij and an end time etij to each operation oprij , that satisfies all problem constraints. Formally, job shop scheduling problems are defined as: Given • Ji , i = 1,…, m, a set of m jobs. • Rk , k = 1,…, n, a set of n resources (with unit capacity). • Ji = {oprijk }, j = 1,…, mi , mi ≤ n. Each job Ji consists of a set of sequential operations. An operation oprijk is the jth operation in the job Ji and requires the use of resource Rk for a processing time of pij . • rti , i = 1,…, m, a set of release times, where rti is the release time of Ji ..

(6) J.S. Liu, K.P. Sycara y Agile manufacturing. 240. Find stij , i = 1,…, m, j = 1,…, mi , such that • rti ≤ sti1, and • stij ≥ eti( j –1) (or equivalently, stij ≥ sti( j –1) + pi( j –1), and k • (etpq ≤ st kij) ∨ (et kij ≤ st kpq), p ≠ i.. A schedule can be visualized using Gantt charts, where operations are organized along the time line from the perspectives of either jobs or resources. Each box represents an operation with length proportional to its processing time and is marked by a unique identification code. We use a three-digit code, where the first digit is the job number, the second digit is the operation number within the job, the third digit is the B represents the first operation resource that processes the operation. For example, opr 21 of J2 and is processed by RB . Figure 1 shows the job Gantt chart of a schedule where only operation precedence constraints and job release date constraints are considered. A vertical arrow in front of each time line represents the release date of each job. The schedule is infeasible because resource capacity constraints are not respected. For example, according to. Figure 1. Job Gantt chart of an infeasible schedule. B B the schedule, RB will process part of opr11 and opr 31 at the same time, which is not feasible. Figure 2 shows the resource Gantt chart of a schedule where only resource capacity constraints are considered. The schedule is also infeasible because job release date constraints and operation precedence constraints are not respected. For example, C B according to the schedule, RC will process opr 12 before opr 11 has been finished by RB , which is not feasible. Also, RC will not be able to process opr C21 as scheduled because J2 has not been released at time 0.. Figure 2. Resource Gantt chart of an infeasible schedule..

(7) J.S. Liu, K.P. Sycara y Agile manufacturing. 241. We use the above examples to illustrate that when solving job shop scheduling problems, two aspects of constraints that involve job and resource, respectively, must be considered simultaneously. A feasible schedule is shown in figure 3. The job Gantt chart of the schedule displays when the operations of a given job are processed. The resource Gantt chart of the schedule lays out the process of a given resource performing on various operations.. Figure 3. Job and resource Gantt charts of a feasible schedule.. In general, scheduling problems are NP-hard [11]. They require time exponential in the input length. Consequently, the time required to solve problem instances of reasonable size can be on an astronomical scale. For example, consider an exponential relation of time 2n to problem size in the number of operations n, and a time unit of a microsecond. The worst case time required to solve the problem grows rapidly from 0.001 second for size 10 (n = 10), to 18 minutes for size 30, and to 35.7 years for size 50. This exponential explosion can not be alleviated by realistic speed increase of computation. To double the size of a problem that can be solved within 18 minutes would require a 109-fold increase in speed. A practical job shop scheduling problem typically involves hundreds of operations. Therefore, almost all realistic scheduling techniques resort to heuristic search that does not guarantee completeness. Given a job shop scheduling problem, the number of feasible solutions can be enormous. For example, for a problem with m jobs of n operations on n resources, each resource has m! possible processing sequences. The total number of possible schedules is (m!)n since all precedence constraints between operations can be satisfied by right shifting operations toward the end of the time line. We need to specify conditions that restrict admissible schedules. In general, there are two types of scheduling problems, e.g., satisfaction problems and optimization problems. In satisfaction problems, each job is assigned a nonrelaxable due date. An admissible solution is a schedule in which all jobs are finished.

(8) J.S. Liu, K.P. Sycara y Agile manufacturing. 242. before their due dates. All admissible solutions are equally good. Usually, we are looking for any one and only one of the admissible solutions. In optimization problems, feasible schedules are evaluated according to objective criteria that reflect the economic goals of organizations. The best solution is a schedule with minimum objective cost. Usually, it is not realistic to find an optimal schedule 1) because of the enormous numbers of feasible schedules. The practical goal is to find a schedule as good as possible. In this study, we consider both job shop scheduling satisfaction and optimization problems. 2.2. Objective criteria We introduce two objective functions that are most commonly used in the literature of job shop scheduling. First, we describe two primitive measures that are essential for the objective functions that we consider. Completion time Ci is the time at which a job Ji completes its processing. It is the end time of the last operation of Ji . Tardiness Ti is the non-negative amount of time by which the completion time of Ji exceeds its due date ddi . If Ji is completed earlier than ddi , the tardiness is zero. (Ti = max[0, (Ci – ddi )]. We consider two objective functions: Inventory INV is the in-progress inventory cost of jobs.. INV =. m. ∑ invi × (max[Ci , ddi ] − sti1 ),. i =1. where m is the number of jobs, invi is a marginal inventory cost introduced to each job by its first operation, sti1 is the time at which a job Ji begins its processing, or the start time of the first operation of Ji . Weighted tardiness Twt is the sum of proportional tardiness multiplied by job importance. Twt =. m. ∑ wiTi ,. i =1. where wi is the weight of a job Ji . We study job shop scheduling optimization problems based on a cost function that combines both inventory cost and weighted tardiness cost of the schedule. This cost function was proposed in [33] as a more realistic cost model that directly accounts 1). Except for small sized problems and special objective functions, such as makespan..

(9) J.S. Liu, K.P. Sycara y Agile manufacturing. 243. for the tardiness cost, in-process inventory cost, and finished-goods inventory cost introduced by each job. Weighted tardiness cost and inventory cost are usually conflicting objectives.2) In general, it is more difficult to optimize a composite objective than just one objective. We adopt this cost function in order to test the capability of our cooperative problem solving approach to optimize conflicting objective. 2.3. A cooperative problem solving model for job shop scheduling Job shop scheduling problems involve job and resource constraints. Intuitively, it is convenient to assign separate agents that would be in charge of these two aspects of constraints. We propose a model of distributed job shop scheduling in which each job is assigned to a job agent and each resource to a resource agent. Therefore, the number of agents involved in solving a job shop problem is equal to the sum of the number of jobs and the number of resources in the problem. Job agents are concerned with constraints of a job’s aspect, e.g., job release dates, due dates, and operation precedence relations. Resource agents take care of constraints of a resource’s aspect, e.g., resource capacity. These agents engage in a coordinated decision making process to solve a scheduling problem. This model seems to be appropriate for the following reasons. (1) Each agent corresponds to a real entity. This provides a closer management of an entity in its situated environment and, therefore, facilitates potential real-time responsiveness. For example, a resource agent can monitor its local condition (e.g., machine breakdown, processing time variation) and update its processing decisions accordingly. A job agent can respond to due date changes and make necessary revisions on operation processing. (2) Each agent can make decisions based on its own strategies and its local situation. This provides modularity for constructing the system. For example, we may want a resource agent to manage the resource differently depending on its contention level. Similarly, we may want a job agent to act with different strategies according to the job’s importance. (3) This model provides an appropriate level of granularity to disentangle the problem constraints and analyze how they interact with each other. By examining how a job interacts with a resource and scrutinizing how resources interact with each other through jobs, and vice versa, we hope to obtain information that can facilitate our understanding of the interaction structure of job shop and the design of communication messages and coordination strategies that enable more effective problem solving. This model has three unique features for cooperative problem solving. First, each agent is concerned with only one type of constraint among a subset of decision vari2). This composite objective demands just-in-time scheduling, which is one of the irregular objectives [24]..

(10) J.S. Liu, K.P. Sycara y Agile manufacturing. 244. ables (operations). A study based on this model may provide implications to other group decision problems that involve several parties of different interests or concerns. Second, an operation is governed both by a job agent and a resource agent. Changes of operation start times by job agents may result in constraint violations for resource agents and vice versa. Therefore, the start times of operations must be negotiated by job and resource agents. Third, there is no authority among agents. All agents have equal right to assign a value (start time) to a decision variable (operation) under its jurisdiction. Therefore, a solution is a value assignment of all decision variables that all agents agree upon. We consider a cooperative problem solving approach based on an iterative negotiation process of agents’ value assignment to decision variables. Negotiation proceeds by proposal and counter-proposal of value assignment between resource agents and job agents. Resource agents are concerned with resource capacity constraints only. Job agents are concerned with job release dates, due dates, and operation precedence constraints. In optimization problems, job due date constraints are relaxed by job agents. Whenever an agent is not satisfied with the current value assignment, it presents a counter-proposal by actually changing the values of a subset of decision variables to satisfy its own constraints. If an agent is satisfied with the current value assignment, it does not change any value assignment. Within a round of negotiation (or an iteration cycle), resource agents and job agents take turns in being active in the negotiation, e.g., they each have one chance to examine the current value assignment and propose necessary changes. While agents of different types of constraints negotiate alternatively, agents of the same type can be active simultaneously, each working independently. A solution to the satisfaction problems is the result of a successful negotiation process between job agents and resource agents. In optimization problems, one of the resource agents assumes a special role of presenting a proposal of low global objective cost. The approach is called COordinated Negotiation Agents (CONA). 3.. The design of coordination techniques. Since the agents are engaged in an iterative negotiation process, their proposals must be coordinated to facilitate convergence to an agreement. We developed a coordination scheme that exploits task disparity in job shop. We distinguish between coordination procedure and coordination information that constitute a coordination scheme. A coordination procedure is a desired negotiation pattern of agents. Coordination information is the set of exchanged information among agents that affects agents’ proposals and facilitates desired negotiation results. 3.1. Task disparity In many application domains, problems often exhibit special structures that can be exploited to facilitate more effective problem solving. One of the most recurrent.

(11) J.S. Liu, K.P. Sycara y Agile manufacturing. 245. structures involves disparity among subproblems. We define disparity as the condition where some quantifiable characteristics of subproblems exhibit noticeable deviation from the average. In the context of CONA, disparity of subproblems is exhibited by different levels of agents’ flexibility to accept a proposal without constraint violation. For example, if a resource is used only occasionally for a short period of time, chances are the resource agent would not detect a capacity constraint violation when a job agent changes the start time of its operation using the resource. On the other hand, if the resource is a bottleneck resource which is used most of the times for a long interval, a start time change on an operation would most likely create constraint violation for the resource. In other words, agents that are assigned to bottleneck resources are more constrained than agents that are assigned to non-bottleneck resources in the negotiation process. Therefore, there is a disparity between bottleneck resource agents and nonbottleneck resource agents. Besides disparity of ease of accepting proposals in the negotiation process, bottleneck resources also have dominant effects on both the admissibility and the quality of a schedule. For example, figure 4 shows a simple job shop scheduling problem which includes only three jobs, Ji , i = 1, 2, 3, on three resources, Rj , j = x, y, z. Each box is an. Figure 4. Dominant effects of bottleneck resources.. operation, with the first number representing the number of the job and the second number representing its sequence within the job, i.e., opr23 is the third operation of J2. An operation is under the jurisdiction of a job agent and a resource agent. RY is considered as a bottleneck resource because of its high contention. If the problem is a.

(12) 246. J.S. Liu, K.P. Sycara y Agile manufacturing. satisfaction problem, the schedule in (a) is not admissible because J1 is not finished before its due date. If the problem is a tardiness optimization problem, the schedule in (b) is a better schedule than the schedule in (a) since it does not have any tardiness. The decision of RY agent to process opr12 before opr22 is most critical in generating the schedule in (b). 3.2. Coordination procedure In view of the iterative proposal, counter-proposal process based on resolving constraint conflicts of individual agents and the task disparity on bottleneck resources, we developed a coordination procedure that promotes rapid convergence of negotiation by considering the following principles of interaction control. (1) Least disturbance – Each agent’s proposal of actually changing the start time of an operation potentially disturbs others’ constraint satisfaction status. To minimize reciprocal harmful interaction, agents’ proposals should take into account the constraints of others regarding current value assignment. (2) Islands of reliability – A proposal by the most constrained agent is more likely to be part of a global solution than that of less constrained agents. In traditional constraint satisfaction literature [27], there was a notion of a most constrained variable which has the fewest possible values. The intuition was that instantiating the most constrained variable seemed to be more critical than instantiating other less constrained variables; in addition, the instantiation would be more likely to be correct than the instantiation of variables with large domains of possible values. We extend this intuition to CONA, in which a proposal from the most constrained agent seems to be more reliable and, therefore, can serve as an anchor of interaction. This intuition also incorporates domain knowledge of job shop scheduling where the processing sequence on the bottleneck resource has the most impact on the admissibility and quality of the schedule. This policy suggests that group agreement should be reached by a process of negotiation based on islands of reliability, and modifying islands of reliability only when group agreement is heuristically determined as infeasible under the current anchoring proposal. (3) Loop prevention – Looping behaviors, such as oscillatory value changes by a subset of agents, should be prevented. This is a common desired property of all distributed systems. We now give a conceptual overview of how these principles guide coordination of the negotiation agents. 3.2.1. Least disturbance Least disturbance corresponds to an attempt to minimize ripple effects of agents’ proposals of actual value changes. To reach group agreement, the number of.

(13) J.S. Liu, K.P. Sycara y Agile manufacturing. 247. unsatisfied agents within a round of negotiation should be gradually reduced to zero. While an agent always becomes satisfied in an iteration cycle since it assigns values to its variables to satisfy only its own constraints, its proposal may cause constraint violations to other agents. Therefore, an agent should resolve conflicts in a way that minimizes the extent of causing disturbances to other agents. Least disturbance is incorporated in an agent’s heuristics of conflict resolution (see section 4). The least disturbance principle is operationalized during conflict resolution in two ways. First, an agent changes the values of as few variables as possible. Second, for a given selected variable, an agent changes the value assignment such that it deviates from the previous value the least possible. 3.2.2. Islands of reliability An island of reliability is a proposal by the most constrained agent and is more likely than others to be part of the solution. Islands of reliability provide anchoring for reaching group agreement in terms of propagating more promising partial solutions and are changed less often.3) In CONA, the island of reliability refers to start times assignment on operations using the bottleneck resource by the agent in charge of the resource. An island of reliability is used as an anchor of interaction during the iterative negotiation process between job agents and resource agents. Recall that each operation is governed by both a job agent and a resource agent. Therefore, the island of reliability is part of every job agent’s concerns in satisfying its constraints. To use the island of reliability as an anchor of negotiation, we design job agents in such a way that they would not propose to change value assignment on the island of reliability during their conflict resolution unless special conditions occur. Instead, they would propose to resolve their conflicts by changing the value assignment on operations that are not using the bottleneck resource. We use the negotiation efforts between job agents and non-bottleneck resource agents as conditions of when job agents can propose to change value assignment on the island of reliability. To measure the negotiation efforts between job agents and non-bottleneck resource agents, we associate a counter with each operation using the non-bottleneck resources. The counter associated with an operation records the number of times that a nonbottleneck resource agent has changed the value assignment of the operation. In other words, when a non-bottleneck resource agent proposes to change the value assignment of an operation during its conflict resolution, it increases the counter associated with the operation by one. We assign a heuristic threshold of counter value to contain the negotiation efforts of job agents and non-bottleneck resource agents between each. 3). Blackboard systems (e.g., Hearsay-II speech-understanding system [9]) have used the notion of solution islands to conduct an incremental and opportunistic problem solving process. Partial solution islands emerge and grow into larger islands, which it is hoped will culminate in a hypothesis spanning the entire solution structure..

(14) 248. J.S. Liu, K.P. Sycara y Agile manufacturing. configuration of the island of reliability. When a job agent detects an operation with a counter value exceeding the threshold in conflict with an operation using the bottleneck resource, the job agent would change the value assignment of the operation using the bottleneck resource. In response to the proposal by the job agent, the bottleneck resource agent, if it finds itself in conflicts, would propose a new configuration of the island of reliability. All counters are reset to zero by the non-bottleneck resource agents. The job agents and the non-bottleneck resource agents resume their negotiation in the hope of an agreement under the new anchoring proposal. 3.2.3. Loop prevention Under the principles of least disturbance and islands of reliability, the system exhibits only two types of cyclic behavior. First, a subset of job and non-bottleneck resource agents may be involved in cyclic proposals of value changes in order to find an agreement with bottleneck resource agents’ decisions. Secondly, the bottleneck resource agent might propose value changes in a cyclic way. The first type of looping behavior is interrupted by job agents when the counter of a conflicting non-bottleneck operation exceeds a threshold. To prevent the second type of looping behavior, the bottleneck resource agent keeps a history of its value changes so that it does not repeat the same proposal. 3.2.4. Negotiation pattern With the three principles of interaction control, the coordination procedure of the negotiation agents is designed to emulate a hierarchical organization with the bottleneck resource agent(s) on the top level and the job agents and the non-bottleneck resource agents on the bottom level. Job agents interact vertically with the bottleneck resource agent(s) and laterally with the non-bottleneck resource agents. There is no direct interaction between bottleneck and non-bottleneck resource agents. By proposing a configuration of the island of reliability, the bottleneck resource agent(s) exerts constraints to job agents, who then negotiate with the non-bottleneck resource agents for an agreement. When an agreement is heuristically determined as infeasible, job agents break the restriction of the bottleneck resource agent(s) by modifying the configuration of the island of reliability. If a bottleneck resource agent finds itself not satisfied with the configuration of the island of reliability modified by job agents, it would propose a new configuration, which again is regarded as new constraints by job agents. The negotiation process continues until all agents are satisfied, which means that an agreement has been found. The coordination procedure is intended to quickly find an agreement among agents in problems with a very large combinatorial space, such as job shop scheduling, but cannot guarantee that the agents will find a solution if the solution exists. However, in our experimental study, we show that CONA usually converges very fast in problems with noticeable bottlenecks..

(15) J.S. Liu, K.P. Sycara y Agile manufacturing. 249. 3.3. Coordination information In order to coordinate their proposals of value changes, agents need to exchange information. We developed a set of useful coordination information for negotiation agents in job shop scheduling. In DAI research, two types of communication have been used, e.g., message passing and shared memory. Message passing limits unnecessary communication. However, in situations such as tightly coupled problems, where frequent communication is required, message passing results in great computational overhead. Shared memory reduces computational overhead but may become a communication bottleneck and limit decentralization. We judiciously balance the above concerns. We limit inter-agent communication through a scheme that employs efficient, small and distributed shared memories, each of which is associated with and shared by a limited number of agents. In particular, coordination information is associated with each operation and is used to express agents’ views with regard to current value assignment of the operation. Coordination information written by a job agent on an operation is referenced by a resource agent, and vice versa, as shown in figure 5.. Figure 5. Coordination information.. Job agents provide the following coordination information for resource agents: (1) Boundary of an operation is the interval between the earliest start time and latest finish time of the operation (see figure 6). It represents the overall temporal flexibility of an operation within the time window of a job’s release date and due date and is calculated only once during initial activation of job agents. (2) Temporal Slack of an operation is an interval between the current finish time of the previous operation and current start time of the next operation (see figure 7). It indicates the temporal range within which the start time of an operation may be assigned to without causing precedence constraint violations. (This is not guaranteed since temporal slacks of adjacent operations are overlapping with each other.).

(16) J.S. Liu, K.P. Sycara y Agile manufacturing. 250. Figure 6. Coordination information: Boundary.. Figure 7. Coordination information: Temporal Slack.. (3). Weight of an operation is the weighted sum of relative temporal slack with respect to operation boundary and relative temporal slack with respect to the interval bound by the closest bottleneck operation. Denote the boundary of an operation oprij by (st ijb , et ijb ). Denote the temporal slack of oprij as (st ijts, et ijts ). Denote the interval of oprij restricted by the closest bottleneck operation as (st ijbb, etijbb). If there is no bottleneck operation before oprij , then stijbb = st bij. If there is a bottleneck operation oprik before oprij , k < j, then st ijbb = etik + ∑ hj −= 1k pih , where etik is the current end time of oprik . If there is no bottleneck operation after oprij , then etijbb = et ijb . If there is a bottleneck operation oprik after oprij , k > j, then etijbb = stik – ∑ hk −= 1j + 1 pih , where stik is the current start time of oprik . The weight of an operation is defined as. eij = k1 ×. (etijb − stijb ) − (etijts − stijts ) − pij (etijb − stijb ). + k2 ×. (etijts − stijts (etijbb − stijbb ). ,. where pij is the processing time of the operation oprij ; k1 and k2 are adjusting parameters. We arbitrarily set k1 to 10 and k2 to 5. Weight of an operation is a measure of the likelihood of the operation “bumping” into an adjacent operation and an adjacent bottleneck operation, if its start time is changed. Therefore, a high weight of an operation represents a job agent’s preference for not changing the current start time of the operation. In figure 8,.

(17) J.S. Liu, K.P. Sycara y Agile manufacturing. 251. Figure 8. Coordination information: Weight.. both opri3 and oprj 3 are bottleneck operations. Oprj1 of Jj will have a higher weight than that of opri1 of Ji . If both operations use the same resource and are involved in a resource capacity conflict, the resource agent will change the start time of opri1 rather than start time of oprj1. Resource agents provide the following coordination information for job agents: (1) Bottleneck Tag is a tag which marks that this operation uses a bottleneck resource. It indicates the status of operation being a bottleneck operation. (2) Resource Slack of an operation is an interval between the current finish time of the previous operation and the current start time of the next operation in the resource’s processing sequence (see figure 9). It indicates the range of the time interval in which the start time of the operation may be changed without causing capacity constraint violations. (This is not guaranteed since source slacks of adjacent operations are overlapping with each other.). Figure 9. Coordination information: Resource Slack.. (3) Change Frequency of an operation is a counter of how many times the start time of this operation set by a job agent has been changed by a non-bottleneck resource agent. Whenever a non-bottleneck resource agent changes the start time of an operation, it increases change frequency of the operation by one. Conceptually, change frequency measures the negotiation effort of job and non-.

(18) 252. J.S. Liu, K.P. Sycara y Agile manufacturing. bottleneck resource agents between each modification on start times of bottleneck operations. In addition, when a non-bottleneck resource agent detects a highly contended resource interval, change frequency can be used by the non-bottleneck resource agent to propagate its decisions on the dynamically arising bottleneck resource intervals 4) by setting the counter to a number larger than some predetermined threshold. We heuristically set the threshold of change frequency to 3. When a job agent encounters a conflicting operation with change frequency larger than 3, it resolves conflict by changing the start time of the bottleneck operation and, therefore, signals to the bottleneck resource agent that a new configuration of the anchor proposal is required. Coordination information among agents is associated with each operation by its responsible agents. When an agent is resolving constraint violations on an operation under its responsibility, the coordination information provided by the other agents that govern the same operation is used in proposing a new value assignment. 4.. Agents’ negotiation heuristics. Agents’ negotiation heuristics attempt to minimize the ripple effects of causing conflicts to other agents as a result of fixing the current constraint violations. Conflict minimization is achieved by minimizing the number and extent of operation start time changes. The design of agents’ negotiation heuristics incorporates the coordination procedure and the use of coordination information. 4.1. Negotiation heuristics of job agent Job agents resolve precedence constraint violations using the following heuristics: (1) Identify conflict pairs of two adjacent operations whose current start times violate the precedence constraint between them (see figure 10). (2) Resolve first conflict pairs involving a bottleneck operation. (3) Change the start time of one operation only to resolve a conflict pair. 4). In job shop scheduling, the notion of bottleneck usually corresponds to a particular resource interval demanded by operations that exceeds the resource’s capacity. Most state-of-the-art scheduling techniques emphasize the capability to identify dynamic bottlenecks that arise during the construction of the solution [33, 38]. In our approach with agents’ dynamic local interaction, we utilize both notions of static (i.e., fixed resource contention ratio) and dynamic bottleneck. Static bottleneck corresponds to a resource within the entire interval specified by the problem. Dynamic bottleneck is a resource interval that is highly contended by operations as a result of job agents’ proposals and is identified by nonbottleneck resource agents..

(19) J.S. Liu, K.P. Sycara y Agile manufacturing. 253. Figure 10. Conflict resolution of job agent.. (4) For a conflict pair including a bottleneck operation and a regular operation, if the change frequency counter on the regular operation is still under the threshold, change the start time of the regular operation; if not, change the start time of the bottleneck operation. (5) For a conflict pair of regular operations, if one of the two operations can be changed within its boundary and resource slack, change that operation; if not, change the operation with lower change frequency. (6) Change the start time of an operation such that it deviates from the previous value the least possible. In figure 10, the conflict pair of opri3 and opri4 will be resolved first since opri3 is a bottleneck operation. If the change frequency of opri4 is still below a threshold, start time of opri4 will be changed by an addition of T2 (the distance between current start time of opri4 and current end time of opri3) to its current start time. Otherwise, start time of opri3 will be changed by a subtraction of T2 from its current start time. In both cases, start time of opri5 will be changed to the end time of opri4. To resolve the conflict pair of opri1 and opri2, either start time of opri1 will be changed by a subtraction of T1 from its current start time or start time of opri2 will be changed by an addition of T1 to its current start time. The decision is based on the boundary, temporal slack, resource slack, and change frequency of both operations. 4.2. Negotiation heuristics of non-bottleneck resource agents Non-bottleneck resource agents resolve capacity constraint violations using the following heuristics: (1) Re-allocate the over-contended resource intervals to the competing operations..

(20) 254. J.S. Liu, K.P. Sycara y Agile manufacturing. (2) Keep changes to the start times of these operations to a minimum. (3) Allocate operations in a sequence based on the weight information associated with them. If their original resource intervals have been preempted by other operations, a most adjacent resource interval within their boundaries is allocated, considering the preference of staying within their temporal slacks. (4) If a conflicting resource interval involves more than three operations, assign high change frequency (higher than the threshold) to these operations after conflict resolution. For example, in figure 11, opra4 was preempted by opre2 , which has higher weight. A most adjacent resource interval is allocated to opra4. In addition, when a resource agent detects a high resource contention during a particular time interval (such as the conflict involving oprc3, oprd1, and oprg1), it allocates the resource intervals and assigns high change frequency (higher than the threshold) to these operations, and thus dynamically changes the priority of these instantiations.. Figure 11. Conflict resolution of non-bottleneck resource agent.. 4.3. Negotiation heuristics of bottleneck resource agents A bottleneck resource agent has high resource contention. This means that most of the time, a bottleneck resource agent does not have resource slack between operations. In job shop scheduling, two operators, exchange(i, j) and right-shift(i, t), have been used [21, 22, 24, 28] to modify the processing sequence of operations on a resource. Exchange(i, j) switches (or swaps) the start times of a pair of operations (i, j). Right-shift(i, t) re-assigns the start time of operation i to a later time t. Bottleneck resource agents resolve capacity constraint violations using the following heuristics: (1) Examine the amount of overlap of conflicting operation resource intervals..

(21) J.S. Liu, K.P. Sycara y Agile manufacturing. 255. (2) If the overlap is considered as small, right-shift operations that follow the changed operation. (3) If the overlap is considered as not small, swap the changed operation with an appropriate operation. In figure 12(i), opre3, oprb3, and oprc3 are all right-shifted to resolve capacity conflicts because the conflicted interval is small and there are slacks before the latest finish time of oprc3. In figure12(ii), the changed oprd3 is swapped with oprb3 since the changed start time of oprd3 is later than the start time of opre3. Resource intervals are. Figure 12. Conflict resolution of bottleneck resource agent.. then allocated to the new sequence – oprb3, opre3, oprd3. The intuition behind the heuristics is to keep the number of changed variables as minimum as possible. Note that the new start time of oprd3 is in the same direction (on the time line) of modification made by a job agent on oprd3. This is to comply with the job agent’s constraints of not being able to process the preceding operations of oprd3 earlier. 5.. Group negotiation process. Given a job shop scheduling problem, agents are created to represent jobs and resources. The group negotiation proceeds as follows: Step 1. Initialize agents..

(22) 256. J.S. Liu, K.P. Sycara y Agile manufacturing. (a) Job agents become active. Each job agent calculates a boundary for each operation under its jurisdiction considering its release and due date constraints. (b) Resource agents become active. • Each resource agent calculates the contention ratio for its resource by summing up the processing times of operations on the resource and dividing it by the interval length between the earliest and latest boundary time among the operations. Denote the set of operations using a resource Rk by {opr ijk }. Denote the boundary of an operation by (st ijb , et bij). A resource’s contention ratio r is calculated by r =. ∑ pij ( max (etijb ). − min( stijb )). ,. where pij is the processing time of an operation oprij . If the resource contention ratio r is larger than a certain threshold, a resource agent concludes that it is a bottleneck resource agent 5) and marks the operations under its jurisdiction as bottleneck operations. • Each resource agent heuristically allocates the earliest free resource interval to each operation under its jurisdiction according to each operation’s boundary. All operations are assigned a start time. This initial value assignment of all variables represents the initial proposal of the tentative schedule. Step 2. Job agents are active. Each job agent looks for constraint violations. If it does not find any, it announces that it is in a satisfaction status. Otherwise, it proposes changes to the current value assignment. Step 3. Resource agents are active. Each job agent looks for constraint violations. If it does not find any, it announces that it is in a satisfaction status. Otherwise, it proposes changes to the current value assignment. Step 4. If all agents are in the satisfaction status, stop the process. The current value assignment of variables represents an agreement of all agents and is a solution to the problem. Otherwise, go to step 2. 5.1. An illustrative example We illustrate, in more detail, the negotiation process through an example. This example features detection of dynamic bottlenecks and reconfiguration of the island of 5). If no bottleneck resource is identified, the threshold value is lowered until the most contended resource is identified..

(23) J.S. Liu, K.P. Sycara y Agile manufacturing. 257. reliability. The input scheduling problem consists of five jobs on five machines with the same due date. During initialization, job agents calculate boundary information for each operation according to the releaseydue dates and the processing times of operations. In figure 13, t1 is the earliest time opr00 can start; t4 is the latest time opr00 should finish if job0 is to meet its due date. Therefore, (t1, t4) is the boundary of opr00 . Denote the boundary start time and boundary end time of opr00 by st b00 and et b00, respectively. We have st b00 = t1 and et b00 = t4 .. Figure 13. Initialization of job and resource agents.. With this boundary information associated with each operation, resource agents build an initial schedule with a heuristic, e.g., allocate free resource intervals to operations with earliest boundary start time at the beginning and then with earliest boundary end time if there is no idleness. For example, in resource0, both opr00 and opr20 have the earliest boundary start time. Opr00 is arbitrarily chosen for allocation. Then, opr20 is allocated next because it has the earliest boundary end time among the remaining operations. The allocation sequence is followed by opr31, opr41, and opr13. Note that opr13 is assigned to its earliest start time. In resource1, after the allocation of opr10 which has the earliest boundary start time, the allocation sequence of opr30 , opr21, opr01, and opr40 is established according to earliest boundary end time. Among the five resources, R2 identifies itself as the bottleneck resource since it has the largest resource contention ratio. All five operations using R2 are marked as bottleneck operations. Figure 14 shows the first cycle of the negotiation process. In job agents, operations represented by two boxes are those involved in constraint violations and whose.

(24) 258. J.S. Liu, K.P. Sycara y Agile manufacturing. Figure 14. Cycle 1 of an example negotiation process.. start times are changed. The dotted boxes represent their previous locations assigned by resource agents. The solid boxes represent their new locations assigned by job agents after conflict resolution. J0 detects two conflict pairs – (opr01, opr02) and (opr02 , opr03). Since opr02 is a bottleneck operation and change frequencies of opr01 and opr03 are zero, both opr01 and opr03 are assigned to new locations (heuristics 3 and 4). Opr04 is also assigned to a new location in order to comply with the conflict resolution between opr02 and opr03. In J3, two conflict pairs (opr30 , opr31) and (opr32 , opr33) are detected. The conflict between opr32 and opr33 is resolved as in J0 . The conflict between opr30 and opr31 can not be resolved within their resource slacks (heuristic 5). Since change frequencies of both operations are zero, J3 heuristically chooses opr30 to change its start time. Resource agents become active after all job agents have made necessary modifications on the tentative schedule to ensure no violation of their constraints. In resource agents, the dotted boxes represent operations’ previous locations assigned by job agents. The solid boxes represent operations’ new locations assigned by resource agents after conflict resolution. In resolving conflicts, resource agents allocate operations to a location as close to the original location as possible (heuristics 1 and 2). In R0 , a capacity constraint violation between opr00 and opr20 is detected. Since opr20 has a higher weight than opr00 , opr00 is changed to a new location (heuristic 3). R1 detects a high resource contention between opr10 , opr21, opr30 , and opr40 . Opr21 and opr01 are allocated to their original locations because they have higher weight information than other operations (heuristic 3). Next, opr30 is allocated to a location just after opr01 because its processing time is longer than the free resource interval between opr21.

(25) J.S. Liu, K.P. Sycara y Agile manufacturing. 259. and opr01 (heuristic 2). The allocation is followed by opr40 and opr10. Opr10 is allocated to the free resource interval between opr21 and opr01 since its processing time is equal or smaller than the interval (heuristic 2). After conflict resolution, the R1 agent assigns a change frequency higher than the threshold to opr10 , opr30 , and opr40 in order to propagate its decisions on the dynamic bottleneck interval (heuristic 4). In R4 , the agent allocates opr34 and opr24 to a later start time, instead of an earlier start time, by considering their temporal slack information (heuristic 3). Figure 15 shows the second cycle of the negotiation process. J1 changes the location of opr11 because opr11 has a high change frequency (heuristic 4). J3 changes the locations of opr31, opr32, and opr33 because opr30 has a change frequency higher than the threshold (heuristic 4). J4 also assigns a new start time to opr41 because. Figure 15. Cycle 2 of an example negotiation process.. opr40 has a high change frequency. J0 and J2 are satisfied with the current solution and make no modification. During resource agents’ negotiation activities, only the R2 (the bottleneck resource) agent finds constraint violation. Since the interval of capacity constraint violation is small, R2 right-shifts opr42 and opr12 (heuristic 2). This represents a new configuration of the islands of reliability. All operations’ change frequency counters are reset to zero. Figure 16 shows the third cycle of the negotiation process. In both J1 and J4 , opr13, opr14 and opr43, opr44 are assigned to a later start time because opr12 and opr42 are bottleneck operations and all these non-bottleneck operations have a change frequency of zero (heuristic 3). R4 is the only resource agent which finds constraint.

(26) 260. J.S. Liu, K.P. Sycara y Agile manufacturing. Figure 16. Cycle 3 of an example negotiation process.. Figure 17. Cycle 4 of an example negotiation process..

(27) J.S. Liu, K.P. Sycara y Agile manufacturing. 261. violation when resource agents become active. R4 assigns new start times to opr34 and opr24 because they have lower weight information than opr43. Figure 17 shows the fourth cycle of the negotiation process. All job and resource agents are satisfied with the current solution. Therefore, the current value assignment is an admissible schedule for the job shop scheduling constraint satisfaction problems. 6.. Negotiation for optimization. Recall that in job shop scheduling, bottleneck resources have the dominant effects on both the admissibility and quality of the schedule (see figure 4). In constraint optimization problems with a tardiness objective, a job’s tardiness is mostly due to its delay on the bottleneck resource. In other words, the operation processing sequence on the bottleneck resource is responsible for most of the tardiness cost of jobs. Therefore, CONA can be naturally extended to the job shop scheduling constraint optimization problems with a tardiness objective. We developed a negotiation mechanism for group optimization, called Anchor& Ascend, where a globally favorable solution is found by iterative local negotiation of an initial proposal based on an anchor sub-solution with monotonically increased local objective cost. In Anchor&Ascend, one of the bottleneck resources is selected and assigned the role of anchor (agent) by heuristic, e.g., assign the role of anchor agent to the last bottleneck in jobs’ processing sequence. The non-bottleneck resources together with the not selected bottleneck resources will be referred to as regular resources. Suppose every uth operation in each job uses the selected bottleneck resource. For each job Ji , we define pre-anchor operations to be {oprij}, j = 1,…, u – 1, the anchor operation to be opriu , and post-anchor operations to be {oprij}, j = u + 1,…, n. A valid sub-solution for the anchor agent is a sequence of the anchor operations {opriu}, i = 1,…, m, with specified start times {stiu}, in which none of the intervals (stiu , stiu + piu) overlaps with others and all stiu ≥ estiu. The local tardiness objective function for the anchor agent is Canchor = ∑ mi=1{wi × max[0, (stiu – lstiu)]} = ∑ mi=1ciu , where ciu is the tardiness cost of opriu , m is the number of jobs, and lstiu = ddi – ∑nk = u pik . The local cost Canchor is used by the anchor agent to evaluate its sub-solution and estimate the global weighted tardiness cost of the current solution. In other words, we assume all the tardiness of jobs is due to tardiness on the bottleneck resource and we use ciu to predict the cost ci of the job Ji by assuming all post-anchor operations in the job will be processed with no delay, i.e., stij = stiu + ∑ kj −= 1u pik , i = 1,…, m, j = u + 1,…, n. Of course, any actual delay will certainly increase ci and increase the overall weighted tardinesscost C, i.e., Canchor ≤ C. Anchor&Ascend expands CONA by incorporating local optimization in the selected bottleneck resource. The anchor agent conducts a local best first search for configuring its local sub-solution. Initially, the anchor agent constructs an optimal local sub-solution by heuristic local search. This anchor sub-solution is used as an island of reliability in CONA for constructing a feasible global solution. If the CONA.

(28) 262. J.S. Liu, K.P. Sycara y Agile manufacturing. session is not successful and the anchor sub-solution is changed by some job agent, the anchor agent employs a set of modification operators to construct new configurations of the anchor sub-solution, and store these potential configurations into a candidate list. Then the anchor agent selects a configuration with the lowest Canchor from the candidate list. A new CONA session begins with the new anchor sub-solution as the island of reliability. The process is repeated until an agreement is found. Anchor&Ascend controls the distributed local negotiation for global optimization by assuming disparity among agents and going through a process of testing the feasibility of constructing a global solution based on different configurations of the anchor sub-solution with monotonically increased objective costs. It does not guarantee optimal solutions. We must emphasize that in job shop scheduling, most techniques do not provide guarantee for optimality in most objective functions. In practice, the goal is to find a high-quality solution with reasonable computational cost. In addition, Anchor&Ascend is not designed as a general mechanism for distributed optimization. It focuses on a subset of job shop scheduling problems, i.e., problems with clear bottlenecks, where it performs effectively. The following subsections present the Anchor&Ascend distributed optimization procedure in more detail. 6.1. Initial optimization of anchor sub-solution Initially, the anchor agent generates a processing sequence {opriu} on the bottleneck resource according to the initial resource allocation heuristics in CONA. Being a highly contended resource, the bottleneck resource usually processes the sequence k k –1 {opriu} without any slack (gap) between adjacent operations, oprpu , oprqu , and k +1 oprru , where the superscripts (k – 1, k, k + 1) denote the processing sequence on the bottleneck resource. In other words, stpu + ppu = stqu , and stqu + ppu = stru . To optimize the sequence {opriu} with minimal Canchor, the anchor agent goes through an iterative process of switching pairs of anchor operations opriu to reduce Canchor .6) During this process, two heuristic subroutines 7) jump forward and jump backward, are used. In jump forward, an anchor operation opriu in the sequence is repeatedly moved forward toward the time origin by switching with one of the preceding operations to reduce Canchor . In jump backward, an anchor operation opriu in the sequence is repeatedly moved backward toward time infinity by switching with one of the succeeding operations to reduce Canchor . Given a sequence of operations, S = {opr iuk }, the subroutines are described as follows: 6). 7). For other objective functions (e.g., makespan), exact methods are available for this one-machine sequencing problem. However, to guarantee optimality for weighted tardiness where no exact method is available, it requires a more elaborate branch and bound procedure which is exponential in the worst case. We choose to rely on heuristics for efficiency. A similar heuristic, pairwise interchange, is also used in neighborhood search [24]..

(29) J.S. Liu, K.P. Sycara y Agile manufacturing. 263. Jump Forward Step 1. Calculate ciu for each opriu. Select an operation, opr υpu , in S with the largest cpu . k Step 2. For each opr iu , k from υ – 1 to 1. If stiu < estpu , go to step 3. Otherwise, if switching oprpu with opr iuk would reduce Canchor , then switch them.. Step 3. Remove oprpu from S. If S is empty, stop. Otherwise, go to step 1. Jump Backward Step 1. Calculate ciu for each opriu. Select an operation, opr υpu , in S with the smallest cpu . k Step 2. For each opr iu , k υ + 1 to m. If stiu < estpu , go to step 3. Otherwise, if switching oprpu with opr iuk would reduce Canchor , then switch them.. Step 3. Remove oprpu from S. If S is empty, stop. Otherwise, go to step 1. To obtain a near optimal sub-solution, the anchor agent repeats the course of applying both the jump forward and jump backward subroutines to the current sequence {opriu} until Canchor can no longer be reduced. 6.2. Modification of anchor sub-solution The changes to a subset of start times {stiu} by a subset of job agents signal to the anchor agent that a reconfiguration of {stiu} is required. The anchor agent uses exchange(i, j) and right-shift(i, t) to modify the processing sequence. In determining the scope of applying the two operators, we consider the following: (1) since the current anchor sub-solution has the minimal local objective cost, it is desirable to modify it as little as possible in order to limit the increase in the objective cost, (2) the modification should also correspond to the changes made by job agents so that new anchor sub-solutions have a better chance of leading to a successful constraint satisfaction iteration. Therefore, right-shift(i, t) is applied to only the changed anchor operation, while exchange(i, j) is applied to a heuristic neighborhood of the changed anchor operation. The local proposal guided by application of the two modification operators generates a set of candidate anchor sub-solutions. These new sequences are put into the list of candidate sequences if they do not duplicate existing sequences in the list. The list of candidate sequences is sorted by increasing local objective cost. Then the anchor agent chooses the first one (with the least cost) from the list to be the next anchor sub-solution and the process of constraint satisfaction by the non-anchor agents repeats until a global solution is found where all constraints are satisfied. Since the anchor agent searches for a proper sequence with monotonically increased objective cost, the global solution represents the best solution that the Anchor&Ascend procedure can find..

(30) 264. J.S. Liu, K.P. Sycara y Agile manufacturing. In particular, suppose the anchor agent has a current operation processing sequence of (A,B,C,D,E,F). If B was changed by a job agent to a later start time t ′, the anchor agent first restores its original valid sequence by changing start times of those changed by job agents back to their original start times. Then the anchor agent applies right-shift(B, t ′) to the current processing sequence. This results in a new sequence with a gap between A and B, and all operations after B are right-shifted accordingly with B. The anchor agent also applies exchange(i, j) to the current processing sequence, with i = B, C, j = i + 1, i + 2, and i = D, j = i + 1. In other words, five exchange operations, exchange(B,C), exchange(B,D), exchange(C,D), exchange(C,E), and exchange(D,E), are individually performed on the current processing sequence, which results in five new sequences, e.g., exchange(B,C) results in (A,C,B,D,E,F), exchange(C,D) results in (A,B,D,C,E,F). These six applications of the two modification operators (five exchanges and one right-shift) represent a heuristic balance between minimizing cost increase (solution quality) and increasing chances of leading to successful search by other agents (search efficiency). In particular, exchange(B,C), exchange(B,D), and right-shift(B, t ′) directly respond to the job agent’s constraint violation (unable to meet constraint imposed by the start time of B) by assigning B to later start times. Exchange(C,D), exchange(C,E), and exchange(D,E) attempt to change the condition of resource contention of regular resources such that the preceding operation of B in the job might start earlier and B might start at its original start time. For example, if one of the preanchor operations of C was competing with the preceding operation of B for the same resource such that the preceding operation of B could not finish before B’s start time, by exchanging C with D and moving C to a later start time, the resource may schedule the preceding operation of B being processed before the pre-anchor operation of C so that B will not be bumped by its preceding operation. When there is more than one anchor operation being changed by job agents, the anchor agent only performs modifications on the earliest changed anchor operation in order to limit cost increase since anchor operations in the later sequence of the anchor agent’s sub-solution have later start times and may have larger tardiness cost than anchor operations in the earlier sequence. 6.3. Algorithmic procedure of distributed optimization The anchor agent is primarily searching for a proper sequence {opriu} with minimal Canchor that would result in a successful negotiation by job and regular resource agents in CONA. Below, we give the algorithmic procedure of Anchor&Ascend. Suppose A is the set of agents and Ac ∈A is the selected anchor agent. Sol( ) is an instance of a (sub)solution generated by an (or a set of) agent(s). OP is a set of modification operations that are designed to generate a neighborhood of Sol(Ac). Si is a state characterized by a Soli (Ac)..

(31) J.S. Liu, K.P. Sycara y Agile manufacturing. 265. Initialization: OLD_STATES, NEW_STATES are empty lists. Step 0. Ac generates and optimizes a sub-solution Sol 0 (Ac). Ac generates a state S0 to record Sol 0 (Ac) and puts S0 in NEW_STATES. Step 1. Ac picks the first state Si in NEW_STATES. This is the current anchor subsolution Sol i (Ac). Step 2. All agents in A – {Ac} engage in a session of negotiation for distributed satisfaction by CONA with Sol i (Ac) anchored. If the session is successful, then an globally high quality solution Sol(A) is found. Terminate the procedure. Otherwise, Ac puts Si in OLD_STATES. Step 3. Ac performs the set of modification operations OP to Sol i (Ac), which results in a set of newly configured sub-solutions {Soli +1(Ac),…, Soli + q (Ac)}. Step 4. For each new Soli + j (Ac), Ac checks if there is an identical instance in either OLD_STATES or NEW_STATES. If not, Ac generates a state Si + j to record Soli + j (Ac). Ac calculates the local objective cost of Soli + j (Ac) as the state cost of Si + j and puts Si + j in NEW_STATES. Step 5. Ac sorts NEW_STATES according to increasing state cost. Go to step 1. The following sections present experimental studies of our cooperative problem solving approach to distributed job shop scheduling. We experimentally compare the results of our work against centralized scheduling techniques. The results show that multi-agent coordination techniques can provide substantial gains in terms of problem solving efficiency and solution quality. 7.. Experimental evaluation in job shop satisfaction problems. We evaluated the performance of CONA on a suite of job shop scheduling satisfaction problems proposed in [33]. The benchmark consists of 6 groups, representing different scheduling conditions, of 10 problems, each of which has 10 jobs of 5 operations and 5 resources. Each group of problems differs in two respects: (1) spread of the release and due dates among jobs; (2) number of a priori bottlenecks. The spread is controlled by varying the amplitude of the intervals within which release and due dates are generated. Three spread levels are introduced: wide (w), narrow (n), and null (0), i.e., both release and due date intervals are collapsed to single points. Aside from different spread levels of release and due dates, the benchmark also considered one and two a priori bottleneck conditions. Each problem in the benchmark has at least one feasible solution. The purpose of this experimental study is to (1) investigate the effects of coordination information in the system, (2) compare CONA’s performance to other constraintbased as well as priority dispatch scheduling methods, (3) investigate the effects of initial solution configuration in the system, (4) investigate CONA’s scaling up characteristics on problems of larger sizes..

(32) 266. J.S. Liu, K.P. Sycara y Agile manufacturing. 7.1. Effects of coordination information In order to investigate the effects of coordination information on the system’s performance, we constructed a set of four coordination configurations. • C0 represents a configuration in which the system ran with no coordination information at all. Without boundary information, when initially activated, resource agents allocate resource intervals according to random sequences. When job agents become active, they resolve conflicts by randomly changing the instantiation of one of the two operations in each conflict pair. Similarly, resource agents resolve conflicts based on random priority sequences. • C1 represents a configuration in which only boundary information is available. Resource agents use this information for heuristic initial allocation of resource intervals. After the initial schedule is generated, no other information is available for conflict resolutions. • C2 represents a configuration in which boundary and bottleneck tag information is available. Resource agents use the boundary information for heuristic initial allocation of resource intervals. Job agents use the bottleneck tag information to bias resolution of conflict pairs. • C3 represents a complete configuration in which all coordination information is provided for resource agents and job agents. The set of configurations of coordination information is shown in table 1, with the additional coordination information for each configuration underlined. Table 2 reports the comparative performance of different configurations on the suite of benchmark problems in terms of the number of problems solved out of the 60 problems and the average iteration cycles to solve a problem. The number of iteration cycles that the system was allowed was limited to 100. If there were still conflicts at cycle 100, the system gave up solving the problem. Since system operations in C0, C1, and C2 have a random nature, they were ran on each problem 10 times. The numbers reported are the average number, e.g. 15.8 out of 60 problems were solved means that there were 158 successful runs among 600 (10 runs for each problem). C3 is deterministic and here each problem was tried only once. C0 was only able to solve 8 problems with an average of 33.3 iteration cycles for solution evolution, while C3 was able to solve all 60 problems with an average of 5.8 iteration cycles. As a light increase in the average iteration cycles in C1 (compared to C0) stems from the fact that while C1 was able to solve twice the number of problems than C0 due to boundary information during initial resource interval allocation, it had no other advantages over C0 to resolve subsequent conflicts. We confirm that adding coordination information enables the system to solve more problems within fewer iteration cycles. The results show the utility of coordination information..