Optimization and Implementation of a Load Control
Scheduler Using Relaxed Dynamic Programming for
Large Air Conditioner Loads
Tsair-Fwu Lee, Member, IEEE, Ming-Yuan Cho, Member, IEEE, Ying-Chang Hsiao, Pei-Ju Chao, and Fu-Min Fang
Abstract—This paper presents the optimization and implemen-tation of a relaxed dynamic programming (RDP) algorithm to gen-erate a daily control scheduling for optimal or near-optimal air conditioner loads (ACLs). The conventional control mode for ACL includes demand control, cycling control, and timer control, to as-sist customers for saving electricity costs. The proposed load con-trol scheduler (LCS) scheme supports any combination of these three control types to save costs optimally during the dispatch pe-riod. Microprocessor hardware techniques were applied to carry out the proposed strategy for realistic application. The Visual C++ language was adopted as the developing tool to carry out the pro-posed work. Field tests of controlling air conditioners located in the campus of National Kaohsiung University of Applied Sciences, Kaohsiung, Taiwan, were tested on-site to demonstrate the effec-tiveness of the proposed load control strategy. The results show that interruptible load scheduling can reduce the system load ef-fectively, and the load capacity reduced by the proposed load con-trol strategy follows closely the trajectory of the peak load.
Index Terms—Implementation, load control scheduler, opti-mization, relaxed dynamic programming.
I. INTRODUCTION
P
OWER energy is the foundation stone of industrial de-velopment. However, the demand for electrical power is growing rapidly, especially in regions where industrial and com-mercial activities are developing fast, so energy saving has be-come an important issue. The annual report on electricity load growth in Taiwan published by the Tai-Power Company (TPC), the sole provider of electricity in Taiwan, shows that approxi-mately 65% of the total power consumption in Taiwan is used by industry and commerce. Efficient use of energy is of ongoing concern, especially for industries with high levels of power con-sumption, such as the iron and steel, petrochemical, cement, andManuscript received January 16, 2007; revised September 25, 2007. Paper no. TPWRS-00018-2007.
T.-F. Lee and F.-M. Fang are with the Department of Radiation Oncology, Chang Gung Memorial Hospital Kaohsiung Medical Center, Kaohsiung, Taiwan, R.O.C. (e-mail: [email protected]; [email protected]). M.-Y. Cho is with the Department of Electrical Engineering, National Kaoh-siung University of Applied Sciences, KaohKaoh-siung, Taiwan, R.O.C. (e-mail: [email protected]).
Y.-C. Hsiao is with the Department of Electrical Engineering, Fortune Insti-tute of Technology, Kaohsiung, Taiwan, R.O.C. (e-mail: [email protected]. tw).
P.-J. Chao is with the Department of Radiation Oncology, Kaohsiung Yuan’s General Hospital, Kaohsiung, Taiwan, R.O.C. (e-mail: [email protected]. edu.tw).
Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TPWRS.2008.919311
paper-making industries, which need to continually consider en-ergy-saving plans to increase their competitiveness [1], [2].
Analysis of the power consumption by commercial customers shows that the operation of air-conditioning equipment is the component with the highest power utilization. Energy-saving systems (controllers) that could aid the TPC and customers in executing various load management schemes and air con-ditioner loads (ACLs) dispatch strategies, such as demand control, cycling control, and timer control, to solve shortages in electricity supply and problems of energy spending during the summer season, are popular topics for studies [3], [4].
In recent years, many successful applications on direct loads control (DLC) have been reported. Bhatnager and Rahman [5] pointed out that the potency of a DLC strategy depends on the system characteristics. Lee and Wilkins [6] presented a general modeling technique to assess the benefits of loads control. Eng-land and Harrison [7] reported the controlling experiences of air conditioners and electrical water heaters in Carolina Power & Light Company. Chen [8] proposes three alternative approaches for the interruptible load control. Wei and Chen [9] proposed the multipass dynamic programming for the DLC; it can converge quickly to the near-optimal solution. To raise the customer’s willingness to attend the DLC, fuzzy logic-based demand con-trol strategy [10] is used for the periodic concon-trol of the elec-trical water heaters and air conditioners. Hsu and Su [11] pre-sented a dynamic programming (DP) method to coordinate the DLC strategies and unit commitment such that the system pro-duction cost could be minimized. The interruptible load control [12]–[19] is an effective load management to reduce the peak load demand on the status of emergency.
In order to implement load management, demand contracts that require an interruptible load are offered to customers at a discount. The peak load can be reduced during periods of high demand by interrupting loads according to the load control scheduling. To keep the inconvenience for customers as small as possible, the load groups should be shed in sequence according to the schedule of the interruptible load groups (ILGs) [12], [13]. In addition to the critical issue of production quality, cost-ef-fective electric energy management is now being considered by customers [12]–[19]. For power energy saving issue, a load con-trol scheduler (LCS) is needed to assist customers in electricity cost saving with minimal discomfort.
The optimal scheduling policy considered here is derived using DP. The optimal sequence of current and future sched-uling decisions should be determined for each initial state. Due to combinatorial expansion, an exact solution can be obtained only for very simple cases. To overcome this problem, Rantzer [20] introduced the technique of RDP, which allows suboptimal
solutions with pre-specified loose bounds to be calculated for switched load control problems. We use those results to solve the optimal large ACLs scheduling problem for direct loads control tasks. This is a typical optimization under constraints problem, in which the objective function to be minimized is the total running cost over the study period subject to the constraints.
In the conventional DP method, the minimum cost-to-target is calculated at each discrete time stage for each possible state at that stage. The large number of state points, called grid points, however, demands an unreasonably large amount of computa-tional time and memory. The RDP was introduced in order to make DP solutions feasible [20]–[22]. The optimal scheduling decision at each time stage is a function of the states of the con-trolled items. In order to solve the scheduling problem for online realistic examples, the factors concerned above are all taken into account and then use the RDP technique to compute optimal or suboptimal solutions with loose criteria to avoid the algo-rithm required for a large look-up table [20]–[23]. The proposed method could satisfy the principle of “the faster the system, the simpler the architecture needs to be.”
The main contributions of this paper are
1) applying the RDP algorithm for control strategy optimiza-tion to generate a daily optimal or near-optimal ACLs con-trol scheduling with energy payback and load uncertainty considered in an LCS;
2) saving electricity cost in incentive rate considered; 3) reducing the perturbation to customers caused by loads
in-terrupted period;
4) considering the customers’ willingness into the problem solving procedure when the parameters of the LCS are set; 5) outperforming the DP in less requirements of storage
memory and computational time.
This paper is organized as follows: Section II introduces the DP background, followed by the main relaxation DP method in Section III. For further details on the theoretical basis, the reader is referred to [23] and [24]. The ACL applications are pre-sented in the following sections. Application of a mathematical optimization approach based on an RDP algorithm to identify an optimum load control model is discussed in Section IV. By using an Intel 16-bit microprocessor to develop a load controller comprising demand control, cycling control and timer control schemes are described in Section V. The intent is to develop a multi-objective function based on RDP with an optimum load strategy to aid customers in avoiding violation of their contract, and to save electricity costs. In Section VI, a case study is pre-sented to demonstrate the RDP performance. Finally, the con-clusions are given in Section VII.
II. DYNAMICPROGRAMMINGALGORITHMREVIEWS
Let be the state of a given system at time , while is the value of the control signal [23], [24]. The system evolves as
(1)
For a given cost function
(2) such that with equality only if , we would like to find an optimal control policy , such that the cost is minimized from every initial state. For a fixed control policy , the map from initial state to the value of (1) is called a value function . The optimal value function is denoted
(3) and is characterized by the Bellman equation
(4) A method commonly used to find the optimal value function is value iteration, i.e., to start at some initial , for ex-ample, , and update iteratively
(5) It is well known that value iteration converges under mild conditions. For a more extensive treatment of basic DP theory, the reader can refer to, e.g., [23] and [24]. However, in the con-ventional DP method, the large number of state points demands unreasonably large amount of computational time and storage memory [20]–[22].
III. RELAXINGDYNAMICPROGRAMMING
The idea of RDP was first proposed by B. Lincoln and A. Rantzer of LTH, Lund University, Sweden, in 2003 [20]–[22]. It reduces the complexity by relaxing the demand for optimality to solve the “curse of dimensionality” prob-lems. The distance from optimality is kept within pre-specified bounds and the size of the bounds determines the computational complexity.
We will follow the Lincoln and Rantzer’s method to find a which fulfills and
(6) for all and . Iterative application of the inequalities gives
(7) for every initial state such that the minima are finite. Moreover, the control law with
achieves
(9) In particular, is a stabilizing feedback law because the lower bound in (6) implies that is a Lyapunov function for the closed-loop system [25]. Usually, and are chosen to satisfy
, for example
(10) (11) With this relaxation of Bellman’s equation, we can search for a solution which is more easily parameterized than . Readers can refer to [20]–[22] for the details of convergence.
A. Relaxed Value Iteration
Given satisfying
(12) define and according to
(13) (14) The expressions for and are generally more com-plicated than for . From this, a simplified which satisfies
(15) is calculated. This satisfies
(16) and the procedure can be iterated. In particular, the lower bound shows that grows with at least as fast as standard value iteration with the step cost .
The iteration of (13)–(15), which we call relaxed value itera-tion, can often be used to find a solution of (6). The (and the ) values are chosen as a trade-off between complexity (time and memory) and accuracy. If and are close to 1, then the it-erative condition (15) becomes close to ordinary value iteration (5), which gives high levels of accuracy and complexity. On the other hand, if the fraction is very large, then the accuracy drops, but (15) can be satisfied with less complex computations. Note that if is chosen as in (10) and (11), the relative error in the
value function defined by and is independent of the number of iterations [20]–[22].
The value function approximations considered will have the form
(17) where is a set of (simple, e.g., linear or quadratic) func-tions on , and the “select” operator selects one of the funcfunc-tions according to some criterion (e.g., “maximum,” “minimum,” or “feasible region”). Moreover, it is essential that also and will have the same form. This expression will be used in the fol-lowing sections.
B. Simple Algorithm to Calculate
We have not discussed how to calculate to satisfy (15) until now. Doing this in an optimal way with respect to com-plexity of the optimization may be a very hard problem. How-ever, we can follow a simple and efficient (albeit not optimal) al-gorithm to obtain for the minimum-type (and conversely maximum-type) parameterization. The algorithm is described in the following procedure (From to , minimum-type) [20].
1) Calculate and from :
Define and such that
and .
2) Let and . 3) If possible, find such that
If not, satisfies (15) Done.
4) Define and go to step 3. 5) Let , and add to the set .
The algorithm simply adds elements from the lower bound until the resulting value function satisfies the upper bound . The resulting value function satisfies (15) by construction. Procedure above can be iterated until the predefined stopping criterion is satisfied.
IV. STRATEGYOPTIMIZATION
A. Problem Formulation
There are two objectives to be implemented for the LCS strategy [10], [14], [15]. The first is to minimize the electricity cost and to maximize customer benefits. The second is to maintain system reliability and to minimize the perturbation of load interruption simultaneously. The proposed model can be built to achieve customer satisfaction and peak load reduction. Due to participation of the customer in the load management, the mathematical model of the problem can be formulated as follows.
1) Objective Function: The purpose of LCS is to reduce the
system peak load by controlling the customers’ air conditioners. For minimizing the disturbance to customers, the objective func-tion of LCS considered in this problem comprises two terms. The first term is to minimize inconvenience and disturbance to
customers during the daily dispatch period, which can be ex-pressed as follows:
(18) where
number of ACLs;
: number of control periods (daily);
capacity of the th ACL group in period (kW); state of the th ACL growth in period if the th ACL group is connected to the system in period ; if the th ACL group is disconnected from the system in period .
The second term considers the willingness of customers to be charged an incentive rate for accepting interrupted control schemes [14]. The objective function of LCS (18), which con-tains the cost component, is revised as
(19) (20) where
incentive weighting function of the th ACL group; deviation of the th ACL group from the average of all incentive rates of the ACLs;
objective weighting capacity; shed by demand control, timer control, and cycling control strategies.
The proposed LCS control mode for ACL can be fulfilled by demand control, cycling control, and timer control, to assist customers for saving electricity costs. can be accomplished by using the following statement:
(21) where
total daily saving capacity for demand control;
total daily saving capacity for cycling control;
total daily saving capacity for timer control.
Max-min can express as “minimal perturbation to customers caused by loads interrupted with maximal loads reduction.”
Since the only key issue in timer control is just time, therefore no further discussion will be made in the following contents.
Constraints considered in the first two terms of objective function in each stage will be discussed in the following paragraphs.
2) Constraints: Apart from fitting the above-mentioned
formulation rules, the proposed algorithm must also satisfy the following constraints. (We take into account the constraints that make the approach superior to the conventional DP method, which assumes that the predicted loads are known exactly; i.e., with no errors.) For instance, the energy payback phenomenon may thus cause a secondary peak load and variation of the load pattern. The LCS schedules the cycling on–off times and duration of the ACL groups, aiming to reduce the peak load as much as possible. The maximal disconnected duration of the ILGs should be constrained to avoiding causing inconvenience to the customer. The minimal online duration of the ILGs should be enough to cover the work that is delayed by the power interruption.
a) Load Constraints: For cycling control, during the
cy-cling off period, only the freezing machine of air conditioners (AC) is off, the other devices such as power fans, circulating pumps, and controller are still on operation. Let be the reduced capacity by LCS operation at time stage , and , be the maximum available capacity of LCS, then
for (22) During the cycling off period, the space temperature will rise, depending on a number of factors such as the weather, the envi-ronmental conditions, the efficiency of the air conditioners, start and stop time of control, control intensities, etc. [3], [10], [17]. Therefore, in the next cycling on period, the load demand will be increased due to the energy payback phenomenon, which is expressed as follows:
(23) where
energy payback ratio appearing at the stage ; energy payback period;
energy payback load in period .
The energy payback ratio represents the ac load shape impact curves. It varies with the weather, the control strategy, the end user’s behavior, etc. The energy payback is a function of the device’s energy demand, because the average device cycle is proportional to the difference between its thermostat setting and the surrounding temperature. Here, the energy payback pattern is considered as follows [10]:
(24) In the demand control operational application, to reflect the uncertainties of load forecasts and load control actions, the com-puting time has to be as fast as possible for an online dynamic
TABLE V
RDP SCHEDULINGRESULTS FOR THECASESTUDIED
TABLE VI
BENEFICIALRESULTS FORRDPANDDP
system operation. Moreover, it shows the load patterns for com-parison. Observing the results reveal that, in the case studied, although the performance of the proposed RDP-LCS might just a nearly optimal solution or with the same optimal solution than DP, but reflecting a trade-off between the memory requirement and computational time consumption for realistic cases. Fortu-nately, they are the top items considered on the real-time con-trolled modalities for practical applications.
VII. CONCLUSION
We have formulated the RDP online scheduling problem into an LCS to obtain the control policy. The optimal or near-optimal online scheduling policy has been tested in a case studied in-volving control of 16 large air conditioners. The following con-clusions can be drawn.
1) The approach is compared to a preliminary DP approach in the case studied involving control of the same ACL group with almost the same results within the optimal solution searching.
2) The computational scheme aids customers in restraining peak load demand and in saving electricity costs more ef-fectively in an online LCS.
3) ACL customers can select the optimum control mode flex-ibly in accordance with actual situations, which depend on the systems uncertainties.
4) RDP algorithm is also suitable for the solution of problems with other types of load control operation in the future due
to its reasonable results, fast calculation speed, and less memory requirement.
5) The Tai-Power Company can use the online LCS to aid in executing various load management projects and dispatch schemes to solve shortages in electricity supply during the summer season.
ACKNOWLEDGMENT
The authors would like to thank the anonymous reviewers for helpful comments on the original manuscript and to Mr. C.-N. Chen for his contributions on the prototype of LCS.
REFERENCES
[1] Bureau Energy, Ministry of Economic Affairs of USA, White book, ch. 4, 1998.
[2] M. L. Chan, Load Management Master Plan for Taiwan Power
Com-pany. Sunnyvale, CA: ML Consulting Group, 1996.
[3] L. Yao, W.-C. Chang, and R.-L. Yen, “An iterative deepening genetic algorithm for scheduling of direct load control,” IEEE Trans. Power
Syst., vol. 20, no. 2, pp. 1414–1421, May 2005.
[4] Lab of load management, Institute of Power Research, TPC, The Tech-nologic Platform of Energy Management and Service for Customers, 2002.
[5] R. Bhatnagar and S. Rahman, “Duect load control: Relationships between electric utility experiences/assessments and system charac-teristics,” IEEE Trans. Power App. Syst., vol. PAS-104, no. 8, pp. 2168–2175, Aug. 1985.
[6] S. H. Lee and C. L. Wilkins, “A practical approach to appliance load control analysis: A water heater case study,” IEEE Trans. Power App.
Syst., vol. PAS-102, no. 4, pp. 1007–1013, Apr. 1983.
[7] W. A. England and L. W. Harrison, “Integrating the load management function into the energy control center,” IEEE Trans. Power App. Syst., vol. PAS-104, no. 6, pp. 1281–1285, Jun. 1985.
[8] D. C. Wei and N. Chen, “Air conditioner direct load control by multi-pass dynamic programming,” IEEE Trans. Power Syst., vol. 10, no. 1, pp. 307–313, Feb. 1995.
[9] H. Salehfar, P. J. Noll, B. J. LaMeres, M. H. Nehrir, and V. Gerez, “Fuzzy logic-based direct load control of residential electric water heaters and air conditioners recognizing customer preferences in a deregulated environment,” in Proc. IEEE/Power Eng. Soc. Canada, 1999, pp. 1055–1060.
[10] Y. Y. Hsu and C. C. Su, “Dispatch of direct load control using dynamic programming,” IEEE Trans. Power Syst., vol. 6, no. 3, pp. 1056–1060, Aug. 1991.
[11] Y. C. Chang and H. C. Tu, “An effective method for reducing power consumption-optimal chiller load distribution,” in Proc. IEEE Power
Control, 2002, vol. 2, pp. 1169–1172.
[12] H.-T. Yang and K.-Y. Huang, “Direct load control using fuzzy dy-namic programming,” Proc. Inst. Elect. Eng., Gen., Transm., Distrib., vol. 146, no. 3, pp. 123–130, May 1999.
[13] K. D. Le, P. F. Boyle, M. D. Hunter, and K. D. Jones, “A procedure for coordinating direct load control strategies to minimize system pro-duction costs,” IEEE Trans. Power App. Syst., vol. PAS-102, no. 6, pp. 1843–1849, Jun. 1983.
[14] K. Y. Huang, “Demand subscription services—An iterative dynamic programming for the substation suffering from capacity shortage,”
IEEE Trans Power Syst., vol. 18, no. 2, pp. 947–953, May 2003.
[15] K. Y. Huang and Y. C. Huang, “Integrating direct load control with interruptible load management to provide instantaneous reserves for ancillary services,” IEEE Trans. Power Syst., vol. 19, no. 3, pp. 1626–1634, Aug. 2004.
[16] L. A. Tuan and K. Bhattacharya, “Competitive framework for procure-ment of interruptible load services,” IEEE Trans. Power Syst., vol. 18, no. 2, pp. 889–897, May 2003.
[17] Y. Y. Hsu and C. C. Su, “Fuzzy dynamic programming: An applica-tion to unit commitment,” IEEE Trans. Power Syst., vol. 6, no. 3, pp. 1231–1237, Aug. 1991.
[18] S. Majumdar, D. Chattopadhyay, and J. Parikh, “Interruptible load management using optimal power flow analysis,” IEEE Trans. Power
Syst., vol. 11, no. 2, pp. 715–720, May 1996.
[19] C. S. Chen and J. T. Leu, “Interruptible load control for Taiwan power company,” IEEE Trans. Power Syst., vol. 5, no. 4, pp. 460–465, Nov. 1990.
[20] B. Lincoln and A. Rantzer, “Relaxing dynamic programming,” IEEE
Trans. Autom. Control, vol. 51, no. 8, pp. 1249–1260, Aug. 2006.
[21] B. Lincoln and A. Rantzer, “Suboptimal dynamic programming with error bounds,” in Proc. 41st Conf. Decision and Control, 2003. [22] A. Cervin and P. Alriksson, “Optimal on-line scheduling of multiple
control tasks: A case study,” in Proc. 18th IEEE Euro micro Conf.
Real-Time Systems (ECRTS’06), 2006.
[23] R. E. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.
[24] D. P. Bertsekas, Dynamic Programming and Optimal Control, 2nd ed. Belmonth, MA: Athena Scientific, 2000.
[25] A. Rantzer, “A dual to Lyapunov’s stability theorem,” Syst. Control
Lett, vol. 42, no. 3, pp. 161–168, 2001.
[26] Robert and Sedgewick, Algorithms in C++. Reading, MA: Addison-Wesley, 1990.
Tsair-Fwu Lee (M’03) received the M.S. degree from the National Kaohsiung
First University of Science and Technology, Kaohsiung, Taiwan, R.O.C., in 2003 and the Ph.D. degree in electrical engineering from the National Kaoh-siung University of Applied Sciences in 2007.
He is now an Adjunct Assistant Professor with the National Kaohsiung Uni-versity of Applied Sciences and also a chief of engineers in the Department of Radiation Oncology, Chang Gung Memorial Hospital, Kaohsiung. His major research interests are evolutionary optimization and artificial intelligent algo-rithms for engineering applications.
Ming-Yuan Cho (M’92) has been with National Kaohsiung University of
Ap-plied Sciences, Kaohsiung, Taiwan, R.O.C., since 1992, where he is now a Pro-fessor in the Electrical Engineering Department. From 2000 to 2003, he served as a Chairman of the Electrical Engineering Department. Since 2004, he worked as a Dean in the continuous and extended education office. His research interests include distribution automation, demand side management and control, appli-cations of artificial intelligent to power system, and renewable energy control.
Ying-Chang Hsiao received the M.S. degree from the Energy Technology
Di-vision of the Asian Institute of Technology, Bangkok, Thailand. Currently, he is pursuing the Ph.D. degree from the Department of Electrical Engineering, Na-tional Kaohsiung University of Applied Sciences, Kaohsiung, Taiwan, R.O.C.
He is now a Lecturer with the Department of Electrical Engineering, Fortune Institute of Technology, Kaohsiung. His major research interests are applica-tions of artificial intelligent to power systems.
Pei-Ju Chao received the M.S. degree from the Department of Computer and
Communication Engineering, National Kaohsiung University of Applied Sci-ences, Kaohsiung, Taiwan, R.O.C., in 2003.
She is now with the Department of Radiation Oncology, Kaohsiung Yuan General Hospital. Her major research interests are digital signals processing and medical applications.
Fu-Min Fang received the M.D. degree from Taipei Medical University, Taipei,
Taiwan, R.O.C., and the Ph.D. degree from the Graduate Institute of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan.
He is now an Associate Professor and the Chief Attending Physician of Radi-ation Oncology in Chang Gung Memorial Hospital, Kaohsiung Medical Center. He has been a Visiting Assistant Professor in the Department of Experimental Radiation Oncology at UT-MD Anderson Cancer Center, Houston, TX. He has published more than 50 papers, with interests in the clinical practice and tech-nical evolution of intensity modulated radiotherapy in cancer patients.
