Solve Multiprocessor Real-Time Scheduling Using Competitive Slack Neural Networks
全文
(2) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. job with a range from 1 to N+1. The total number of job is N. The (N+1)th job is a pseudo-job, which is a supplementary job used to facilitate satisfying 1-out-of-N rule. Restate, by adding neurons to the hypothesis representation neurons, we can enforce inequality constraints. The extra neurons are referred to as “slack neurons” herein. In this investigation, slack neurons are those neurons in representing the pseudo-jobs. Meanwhile, the machine variable, j, represents a dedicated machine from 1 to M, the total number machines to be operated. Finally, the time variable, k represents a specific time, which is less than or equal to T, the deadline of the job. Thus, a state variable Vijk is representing whether or not job i is executed on machine j at a certain time k. In addition, the activated neuron Vijk=1 denotes that the job i is run on machine j at the time k; otherwise, Vijk=0. The activated neuron V(N+1)jk=1 indicates processor j at a certain time k is free. Notably, each Vijk corresponds to a neuron of the neural network. The correlating energy function representing the scheduling problem is as Eq. (1). In this equation C1, C2, C3, C4, C5 and C6 refer to weighting factors; N denotes the total number of processes to be scheduled; M is the total number of machines to be operated; T represents the maximum time quantum of a process. These weighting factors, N, M, and T, are assumed to be positive constants herein. The C1 energy term confines a processor j to executing only one process, say i or i1, at a certain time k. This energy term has a minimum value of zero when satisfying this constraint, which occurs when Vijk or Vi jk equals zero.. multi-processor real-time scheduling to meet deadline requirement by applying the k-out-of-N rule to a neural network, slack neurons are extended to agree with the inequality constraints. Tagliarini, Christ and Page [13] demonstrated a weapon-to-target assign problem, a resource allocation task problem. There is a slack neuron associated with each weapon. The slack neurons are used only to assure that problem constraints are satisfied. In most situations, a fully multiprocessor utilization system is a restrictive situation. Moreover, real-time scheduling is interesting on meeting task timing constraints rather than optimizing a given target. In light of above developments, this work investigates the job schedule problem of multi-process on a “fully” or “non-fully” utilized real-time multiprocessor system that includes timing constraints. In addition, this studied problem is presented using a 3-dimentional neural networks. Extra slack neurons are added on to the networks to content the non-fully utilized conditions. An energy function designed to illustrate the timing constraints is proposed. According to the CHNN, the scheduling problem is considered a minimization of an energy function. The simulations involve the fully utilize multiprocessor real-time scheduling problems, which were solved with competitive scheme in [11]. Furthermore, some fully and non-fully utilized real-time scheduling problems were simulated. Notably, the slack neurons should not be taken into account when retrieving the solution of the problem.. 1. 2. Energy function of the scheduling problem. C N +1 M T N +1 E = 1 ∑ ∑ ∑ ∑ VijkVi1 jk 2 i =1 j =1 k =1i1=1 ,i1≠ i C2 N M T M T + ∑ ∑ ∑ ∑ ∑ VijkVij1k1 2 i =1 j =1 k =1 j1=1 k1=1 , j1≠ j C3 N +1 M T 2 + ∑ ( ∑ ∑ Vijk − Pi ) 2 i =1 j =1 k =1 C N +1 M T + 4 ∑ ∑ ( ∑ Vijk − 1) 2 2 i =1 j =1 k =1 C5 N +1 M T 2 + ∑ ∑ ∑ Vijk G ijk H (Gijk ) 2 i =1 j =1 k =1 C N +1 M T M T − 6 ∑ ∑ ∑ ∑ ∑ VijkVij1k1 2 i = N +1 j =1 k =1 j1=1 k1=1 , j1≠ j. The scheduling problem domain to be considered in this paper is defined as follows. Suppose there are N jobs (or processes) and M machines (or processors). First, a job can be segmented and the execution of each segment is preemptive. Second, different segments of a job cannot be assigned to different machines, implying that no job migration is allowed between machines. Third, the execution time and the deadline of each job are predetermined. Furthermore, the processor is allowed to be non-fully-utilized. Based on these assumptions, we attempt to obtain a set of job schedules. Restated, the preemptive processes in a multiprocessor real-time system are interesting. To resolve this problem, an energy function that represents the scheduling problem has to be defined. The defined energy function is transformed into an extended 3-D HNN; then the optimization process searches for neuron states satisfying a set of constraints such that the energy function is minimized. According to the problem, scheduling involves three variables: job (or process), machine (or processor), and time. Thus, neuron states variable Vijk is defined. The job variable, i, indicates a specific. . (1). Gijk = k − di. 1, if .Gijk > 0 where H (Gijk ) = . 0, if .Gijk ≤ 0. The C2 energy term indicates that a process migration is prohibited, implying that process i runs on processor j or j1. This item also has a minimum value of zero when Vijk or Vij1k1 is zero. In the C3 energy term, Pi denotes the total execution time required by process i. This energy term means that. 400.
(3) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. the time consumed by process i must equal Pi such that ∑ ∑ V = Pi , i.e. this term becomes zero. The ijk. th. processing time of the pseudo-job (the N+1 job) is defined as the total available time for all machines subtracts the total processing time required by all N jobs. Additionally, the C4 energy term is to prevent no process being executed on a specific processor at a certain time when 1-out-of-N rule applied. Thus, this energy item falls to a minimum of zero when satisfying this constraint. Meanwhile, the C5 energy term is real-time requirement to meet the deadline of each process i, where di is the time limitation of process i and H(Gijk) is the Heavside function. When a process is allocated with a run time that exceeds d, the energy term will exceed zero, and the energy value will grow exponentially with the associated time lag between di and k. On the contrary, this energy term will have a value zero as long as Vijk=1and k-di ≤ 0. Finally, the C6 term excludes the slack neurons from the 2nd term. Hence, this term allows slack neuron of the N+1th job to be activated on different processors (j and j1 ≠ j ) at any time k1. Since the slack neurons are not to be taken into account when retrieving the solution of the problem. Restate, the 2nd term limits exactly one processor j to run process i, and this term allows more than one processor to run process N+1 (i.e., pseudo-job). Based on the above discussion, the derived energy function has a minimum value of zero when all constraints are satisfied. Eq.(1) can be proved to be an appropriate Lyapunov function for the system under discussion.. 3. Competitive networks. algorithm. of. neural. Hopfield and Tank originally proposed the neural network, HNN, in [14]. Essentially, the HNN algorithm is based on the gradient technique, thus providing rapid convergence. Moreover, the HNN also provides potential for parallel implementation. Based on dynamic system theory, the Liapunov function [14] [15] expended to a 3-dimentional model as shown in Eq. (2), has verified the existence of stable states of the network system. This energy function representing the scheduling problem must be in the same format as the Lyapunov function. 1 E = − ∑ ∑ ∑ ∑ ∑ ∑VxyzWxyzijkVijk + ∑ ∑ ∑θijkVijk 2x y z i j k i j k. 1, if n +1 n Vijk = Vijk , if 0, if . Meanwhile,. Netijk > 0 Netijk = 0. .. (3). Netijk < 0 Net ijk. represents the total input or. net value of the neuron (i, j, k) obtained using the interconnection strength, Wxyzijk , and the bias input, θijk displayed as follows: Netijk = −. ∂E = ∑ ∑ ∑WxyzijkVxyz − θijk ∂Vijk x y z. (4). Instead of applying conventional deterministic rules to update neuron states, competition among neurons is used to decide the winning neuron, i.e. the active neuron in competitive scheme. Restated, the competitive rule is to build neural networks satisfying constraints of the type of “exact one neuron among N” should be activated when the network reaches a stable state, and can be considered as a 1-out-of-N confine rule. Since a processor can only execute one job at a time in subject scheduling problems, omitting the C1 and C4 energy terms from the HNN energy function (Eq. 1) yields a simplified energy function and satisfies the competitive constraint. Restated, the C1 and C4 energy terms are handled implicitly. The resulting energy function for CHNN is highlighted as follows: C 2 N +1 M T M T ∑ ∑ ∑ ∑ ∑ VijkVij1k1 2 i =1 j =1k =1 j1=1 k1=1 j1≠ j 1 N M T + C3 2 + ∑ ( ∑ ∑ Vijk − Pi ) 2 i =1 j =1k =1 C 5 N +1 M T 2 + ∑ ∑ ∑ Vijk Gijk H (Gijk ) 2 i =1 j =1k =1 C N +1 M T M T − 6 ∑ ∑ ∑ ∑ ∑ VijkVij1k1 2 i = N +1 j =1k =1 j1=1 k1=1 , j1≠ j E=. (5). The resulting energy function makes it apparent that this must be an appropriate Lyapunov function. Comparing Eq.(5) with Eq.(2) makes it possible to determine synaptic interconnection strength, Wxyzijk , and the bias input,. θijk. , as illustrated below:. Wxyzijk = −C 2 * δ ( x, i ) * (1 − δ ( y, j )) − C 3 * δ ( x, i ) + C 6 * δ ( x, N + 1) * δ (i, N + 1) * (1 − δ ( y, j )) (6). and. (2). θ xyz = −C 3Pi +. Where Vxyz and Vijk denote the neuron states, Wxyzijk represents the synaptic weight indicating the interconnection strength among neurons, and θ ijk is. C5 * G 2 * H (G ) 2. (7). respectively, where 1 if .a = b 0 if .a ≠ b. δ ( a, b) = . the threshold value representing the bias input of the neuron. Additionally, the HNN employs the deterministic rule to update the neuron state change. This deterministic rule is displayed in Eq. (3) below:. is the Kronecker delta function.. The neurons on the same column of a dedicated processor at a certain time compete with one another to decide which specific job should be the winning neuron. The neuron that receives the maximum net value is the winning neuron. Accordingly, the output of the winner neuron is set to 1, and the output states. 401.
(4) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. of all the other neurons on the same column are set to 0. The winner-take-all update rule of the neuron for the ith column is illustrated as follows: 1 if Vxjk = 0. where. Net xjk = Max Netijk. Net xjk. i =1~ N +1. otherwise. ,. supposed to be the only neuron activated with the largest total input value after updating, that is, V (t +1) = 1 , and qmn (t +1) =0 , for Vimn. (8). The active neuron, based on the winner-take-all update rule as in Eq.(8), is the one with maximum net value on each column in each update, that is Netqmn = Max Netimn . This implies that. is the maximum total neuron input as. in Eq.(4). The ith column consists of jobs 1 ~ N+1 (slack neuron). Restated, the slack neuron can be the wining neuron while satisfying the constraints.. i =1~ N +1. Netqmn > Netlmn. Netqmn = −. This section demonstrates a mathematical proof of convergence in the CHNN for the investigated problem. According to Eq. (4), the neuron (i, j, k) obtains the total input, i.e. net value, which is as follows (Eq.9):. C2 M T ∑ ∑ Vqj1k1 − C 3(Vqmn − Pq ) 2 j1=1 k1=1. Netlmn = −. C2 M T ∑ ∑ Vlj1k1 − C 3(Vlmn − Pl ) 2 j1=1 k1=1 j1≠ m. the summation of the energy term corresponding to neuron state Vimn. The energy function can then be represented as follows (Eq. 10):. Thereby, the energy change difference cab be rewritten as follows: t +1 t ∆E = Emn − Emn M T C2 M T = ( ∑ ∑ Vqj1k1 − ∑ ∑ Vlj1k1) 2 j1=1 k1=1 j1=1 k1=1 j1≠ m. T. ∑ VijkVij1k1). i =1 j =1 k =1 j1=1, k1=1 j ≠ m k ≠ n j1≠ j , j1≠ m N +1 M T C 3 N +1 + ( ∑ (Vimn − Pi ) 2 + ∑ ( ∑ ∑ Vijk − Pi ) 2 ) 2 i =1 i =1 j =1 k =1 j ≠m k ≠n C 5 N +1 2 + ( ∑VimnGimn H (Gimn ) 2 i =1 N +1 M T 2 + ∑ ∑ ∑ Vijk Gijk H (Gijk )) i =1 j =1 k =1 j ≠m k ≠n C6 N +1 M T + ∑ ∑ ∑ (−Vijk )Vij1k1 2 i = N +1 j1=1 k1=1 , j1≠ j. (12). (t ) Vimn = 0(i ≠ l ) .. C 2 N +1 M T ( ∑ ∑ ∑ VimnVij1k1 2 i =1 j1=1 k1=1 ∑. .. Investigating Eq.(10), the total energy difference of the neural network , ∆E , between the (t+1)th iteration and the (t)th iteration is the same as the Emn change between the (t+1)th iteration and the (t)th iteration. Restate, the Eother is canceled out. (t +1) (t ) (t +1) Since Vqmn = 1 , Vimn = 0(i ≠ q ) , Vlmn = 1 , and. remainder, that is, E other . In other word, E mn is. ∑ ∑. (11). C5 2 − G H (Glmn ) 2 lmn. For clarity, we separate this energy function into two parts. The first one is for the processor m at given time n, that is, E mn . The second part is the. j1≠ m T M. are obtained. Netlmn. and. j1≠ j. j1≠ m. +. N N C3 ((1 − Pq )2 − (1 − Pl )2 + ∑ Pi 2 − ∑ Pi 2 ) 2 i =1 i =1. +. C5 2 2 (Gqmn H (Gqmn ) − Glmn H (Glmn )) 2. i≠q. (13). i ≠l. Accordingly, the energy changes between neuron update equals the net value change minus C3. That is ∆E = Net lmn − Net qmn − C 3 .. (10). Obviously, the above equation implies that the energy difference in the update is negative, i.e. ∆E < 0 .Hence, the system is convergent during network evolution. Apparently, this energy function is an appropriate Lyapunov function.. E = E mn + E other . Vimn is the neuron on the ith row (job) and the nth column (time) for the specific processor m. Focusing on these terms at the (t)th iteration, the Vlmn is supposed to be the only active neuron (l,m,n) in the nth column on the processor m before updating, that is, V (t ) = 1 , and lmn (t ) = 0 , for Vimn. and. j1≠ m. (9) C2 M T C5 2 =− G H (Gijk ) ∑ ∑ Vij11k1 − C 3(Vijk − Pi ) − 2 j1=1 k1=1 2 ijk. + ∑. Netqmn. C5 2 − Gqmn H (Gqmn ) 2. ∂E Netijk = − ∂Vijk. N +1 M. .. based on Eq.(9) as follows:. 4. Convergence of the energy function. E=. i ≠ q.. 5. Simulation examples and results A number of sets of timing constraints and various weighting factors were applied to the simulations. The constants in Eq.(11) are set to C2=1.355, C3=0.55, C5=1.355 and used in simulation. The C6 term exclude the slack neurons from C2 term. Consequently, C6 value is equal to the C2 value. Tables 1 through 4 list timing constraints of simulation cases, respectively. The simulation involves scheduling four or five processes (jobs) in. i ≠ l.. Moreover, the neuron (q,m,n) at (t+1)th iteration is. 402.
(5) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. two processors (machines) system. Moreover, more complicated simulation case with10 processes in three processors system is included as well. Case 1a and case 3a are the same conditions as in [11]. They are added to facilitate the full processor utilization system scheduling study by applying the proposed scheme herein. Figs. 1 and 2 illustrate the resulting schedules of case 1a and case 1b for proposed algorithm. Meanwhile, Figs 4 and 5 display the simulation results of case 3a and case 3b.. Finally, Figs. 3 and 6 represent the resulting schedules correlating with the case 2 and 4 respectively. The process assignment of S as displayed in figures 2, 3, 5, and 6, indicates the active “slack” neurons. Restate, the machine at that time does nothing. Additionally, figure 7 displays the significant portions of the energy curves of all cases during neural network evolution.. 6. Discussion and conclusions The competitive mechanism eliminates the constraint terms in the energy function, simplifying the network by reducing the interconnections among neurons [8], as displayed in Eq. (5). This investigation illustrated an approach to mapping the problem constraints into the energy function of the competitive neural networks containing slack neurons, they are involved so as to resolve the timing constraints schedule problem for both “non-fully utilized” and “fully utilized” system. The conventional deterministic rule in determining the neuron state was replaced by winner-take-all or competitive rule. The weighting factor determination is an intrinsic shortcoming of Hopfield neural network and the simulations also encounter this drawback. However, the set of weighting matrix used in our simulation is not unique. Different sets of weighting factors may produce different neural network revolutions. The energy function proposed herein works efficiently and can be applied to the similar cases of investigated scheduling problems. The competitive scheme combined with slack neurons suggests the way to be applied to this kind of scheduling, which has inequality constraints. Restated, solving “fully” or “non-fully” utilized real-time scheduling problems can apply proposed competitive slack neural networks. This work focuses mainly on solving real-time scheduling without ready or setup time consideration. For more practical implementation, different and more complicated scheduling problems can be further investigated in future work by applying this proposed algorithm. Such problems include preemptive multi-process scheduling on multiprocessor system with multi-constraints such as deadline and resource constraints. The problem can be further extended to involve the temporal relationship of ready time or priority for each job. A. future work should address these issues more thoroughly. Acknowledgements: This work is supported by National Science Council (NSC93-2213-E167-012). Reference [1] Willems T. M. and Rooda J. E., “Neural Networks for Job-shop Scheduling,” Control Eng. Practice, 2(1) (1994) pp. 31-39. [2] Foo YPS, Takefuji Y. Integer linear programming neural networks for job-shop scheduling. In: IEEE Int. Conf. on Neural Networks, vol. 2, 1998, pp. 341-348. [3] Zhang CS, Yan PF, and Chang T. Solving Job-Shop Scheduling Problem with Priority Using Neural Network. In: IEEE Int. Conf. on Neural Networks, 1991, pp. 1361-1366. [4] Silva MP, Cardeira C, and Mammeri Z. Solving real-time scheduling problems with Hopfield-type neural networks. In: EUROMICRO 97 'New Frontiers of Information Technology, Proceedings of the 23rd EUROMICRO Conference, 1-4 Sept. 1997, pp. 671 –678. [5] Hanada A, Ohnishi K. Near optimal jobshop scheduling using neural network parallel computing. In: Int. Conf. on Proc.Industrial Electronics, Control, and Instrumentation, vol. 1, 1993, pp. 315-320. [6] Hopfield JJ, Tank DW. Neural computation of decision in optimization problems. Biological Cybernetics 1985; 52: 141-152. [7] Huang YM, Chen RM. Scheduling multiprocessor job with resource and timing constraints using neural network. IEEE Trans. on System, Man and Cybernetics, part B, Aug.1999; 29(4): 490-502. [8] Chung PC, Tsai CT, Chen EL, Sun YN. Polygonal approximation using a competitive Hopfield neural network. Pattern Recognition 1994; 27: 1505-1512. [9] Uchiyama T, Arbib MA. Color Image Segmentation Using Competitive Learning. IEEE Trans. Pattern Analysis Machine Intelligence. 1994; 16(12): 1197-1206. [10] Chen RM, Huang YM. Multi-constraint task scheduling in multiprocessor system by neural network. In: Proc. IEEE Tenth Int. Conf. on Tools with Artificial Intelligence, Taipei, 1998, pp. 288-294. [11] Chen RM, Huang YM. Competitive Neural Network to Solve Scheduling Problem. Neurocomputing, April 2001; 37(1-4):177-196. [12] Cardeira C, Mammeri Z. Neural networks for multiprocessor real-time scheduling. In: IEEE Proc. Sixth Euromicro Workshop on. 403.
(6) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. Real-Time Systems, 1994, pp. 59-64. [13] Tagliarini GA, Christ JF, and Page EW. Optimization Using Neural Networks. IEEE Transaction on Computers Sep. 1991; 40(12): 1347-1358. [14] Hopfield JJ, Tank DW. Computing with neural circuits: A model. Science 1986; 233: 625-633.. [15] Cohen M, Grossberg S. Absolute stability of goal pattern formation and parallel memory storage by competitive neural network. IEEE transaction on system, Man, and Cybernetics Sep/Oct 1983; 13: 815-26. Figure 2. Simulation results of case 1b (non-fully utilized). Table 1. Timing constraints matrix (case 1a: A=4,B=3; case 1b:A=3,B=2). Time Required A 3 B 2. Process 1 Process 2 Process 3 Process 4. Time Limit 6 4 6 3. P4. P4. S. Machine 1 S P2 P2. P2. P2. P1. P3. P3. Machine 2 P3 P1 P1. P1. P1. P4. P1. P1. Machine 1 P4 P4 P4. P5. P5. P3. P3. P3. Machine 2 P2 P2 P2. P2. P2. P1. P4. P1. Machine 1 P4 S P4. S. S. P3. P3. P3. Machine 2 P2 P5 P2. P5. P2. Figure 3. Simulation results of case 2. Table 2. Timing constraints (case 2). Time Required 5 4 3 2. Process 1 Process 2 Process 3 Process 4. Time Limit 8 8 6 3. Figure 4. Simulation results of case 3a (fully utilized). Table 3. Timing constraints (case 3a: A=5,B=4; case2b: A=3,B=3). Time Required 2 A 3 B 2. Process 1 Process 2 Process 3 Process 4 Process 5. Time Limit 3 8 4 8 5. Figure 5. Simulation results of case 3b (non-fully utilized). Machine 1 P9 P8 P6 P8 P6 P9. Table 4. Timing constraints (case 4). Time Limit 10 5 9 5 9 6 10 5 9 10. Machine 1 P4 P1 P1. P4. P1. P2. Machine 2 P3 P2 P2 P3. P4. P2. Machine 2 P2 P3 P2 P3. P9. S. S. Machine 3 P3 P3 P5 P4 P4 P1 P5 P5 P3 P1 Figure 6. Simulation results of case 4. 12. 10. 8. P1. 6. 4 case 2. 2. P3. case 3b. Figure 1. Simulation results of case 1a (fully utilized). Machine 1 P1 P4 P1 P1. S. Machine 2 P10 P2 P7 P2 P2 P7 P10 P7 P10 P10. Energy. Process 1 Process 2 Process 3 Process 4 Process 5 Process 6 Process 7 Process 8 Process 9 Process 10. Time Required 2 3 3 2 3 2 3 2 3 4. 0. -2. S. case 4. case 3a case 1b. case 1a. 0. 2. 4. 6. 8 10 12 # of iterations. 14. 16. Figure 7. Energy function of case 1-4.. S. 404. 18. 20.
(7)
相關文件
Wang, Solving pseudomonotone variational inequalities and pseudo- convex optimization problems using the projection neural network, IEEE Transactions on Neural Network,
SG is simple and effective, but sometimes not robust (e.g., selecting the learning rate may be difficult) Is it possible to consider other methods.. In this work, we investigate
{ Title: Using neural networks to forecast the systematic risk..
CAST: Using neural networks to improve trading systems based on technical analysis by means of the RSI financial indicator. Performance of technical analysis in growth and small
CAST: Using neural networks to improve trading systems based on technical analysis by means of the RSI financial indicator. Performance of technical analysis in growth and small
Random Forest: Theory and Practice Neural Network Motivation.. Neural Network Hypothesis Neural Network Training Deep
They are suitable for different types of problems While deep learning is hot, it’s not always better than other learning methods.. For example, fully-connected
Pascanu et al., “On the difficulty of training recurrent neural networks,” in ICML, 2013..