In this chapter, we established the properties of the induced merit function of three classes of discrete-type NCP functions φpNR, φpS−NR and ψpS−NR. This includes the characterization of stationary points and level sets of the corresponding merit functions.
We have also described the growth behavior of these three families of NCP functions.
Moreover, we proved the Lyapunov and asymptotic stability properties of the proposed steepest descent-based neural networks. Finally, numerical simulations indicate that for
all three families of neural networks, a smaller value of p always yields faster convergence to the NCP solution. However, we note that this is only based on a few set of test NCP examples.
Numerical reports suggest that the neural network based on φpS−NR is capable of outperforming the neural network based on the extensively used FB function, as well as the network based on the generalized FB function. However, it must be noted that as in other neural networks, results may be dependent on the choice of initial conditions.
Nevertheless, it would be an interesting future research endeavor to find out when φpS−NR can be best used and whether it is capable of outperforming other NCP functions when other solution methods are employed. Exploring this could possibly provide an alternative NCP function to use when designing algorithms. Furthermore, the effect of varying the values of p when other approaches are used can also be considered. That is, one could find out if smaller values of p will also yield faster convergence when other solution methods are used. Theoretical proof for performance-dependence on p can also be investigated, which will be pursued in the next chapter in light of the neural network approach. Exploring deeper the properties of φpS−NR deserves particular attention because of its numerical performance, despite its complexity.
A lot of future research directions can be taken on from here since many algorithms in optimization rely on NCP functions. In the case of nonlinear complementarity problems, the three classes of NCP functions used in this chapter can be exploited to design other solution methods such as merit function approach, nonsmooth Newton method, smoothing methods, and regularization approach, among others. The results in this chapter serve as a starting point in designing such approaches, since we have established herein the properties of level sets and stationary points of the induced merit functions.
Indeed, these properties are fundamental in constructing other NCP-functions-based methods. Of course, the NCP functions studied herein can be adopted to formulate neural network approaches to other complementarity problems, as well as variational inequalities and linear and nonlinear programming problems.
Chapter 5
Neural Networks based on Novel Generalization of the Natural
Residual Function
In the previous chapter, three discrete-type families of complementarity functions with parameter p ≥ 3 (where p is odd) based on the NR function were used. Using a neural network approach based on these families, it was observed from some preliminary numerical experiments that lower values of p provide better convergence rates. Moreover, higher values of p require larger computational time for the test problems considered.
Hence, the value p = 3 is recommended for numerical simulations, which is rather unfortunate since we cannot exploit the wide range of values for the parameter p of the family of NCP functions. This chapter is a follow-up study on the aforementioned results. Motivated by previously reported numerical results, we formulate a continuous-type generalization of the NR function and two corresponding symmetrizations. The new families admit a continuous parameter p > 0, giving us a wider range of choices for p and smooth NCP functions when p > 1. Moreover, the generalization subsumes the discrete-type generalization initially proposed. The numerical simulations show that in general, increased stability and better numerical performance can be achieved by taking values of p in the interval (1, 3). This is indeed a significant improvement over preceding studies.
5.1 Motivation
In the preceding chapter, we considered the use of discrete-type generalization and symmetrizations of the natural residual function, namely φpNR, φpS−NR, and ψpS−NR, in constructing the neural network. It was found out from a few numerical simulations that lower values of p yield faster convergence speeds when these three classes of NCP functions are used. Moreover, longer computation time is usually required when a higher value of p is used. There are also test instances when a higher value of p leads to ill-conditioning problems in the numerical simulations. In turn, the choice p = 3 may seem optimal. In other words, the results suggest that choosing higher values of p may not be a good choice. Consequently, this seems to imply that the discrete-type generalization appears to be not very useful in the sense that only one member of each of the families is useful for numerical purposes. This motivates us to explore if there exists a continuous generalization of the NR function, i.e. a generalization parametrized by p which assumes values on some interval. This will provide us more values to consider for the tunable parameter, instead of just the odd integers with value at least 3.
We provide an affirmative answer to this problem. More precisely, the main contribu-tions of this chapter are as follows:
(i) We propose a continuous-type generalization of the NR function. The proposed function does not have a symmetric surface, but we provide two symmetrizations which also admit a continuous parameter p. This generalizes the results in [5, 9].
(ii) We establish several properties of these newly formulated NCP functions which are prerequisite to designing solution methods for the complementarity problem, are not limited to the neural network approach. These properties extend the results in [30].
(iii) Stability properties of the neural network (3.9) will be established as important extensions of the results in [3], i.e. the results in Chapter 4.
More importantly,
(iv) We illustrate that the proposed continuous generalization is meaningful. In partic-ular, it provides a wider range of values of p which offer better convergence speeds than the ones based on the discrete-type generalization and their symmetrizations illustrated in the preceding chapter.
(v) We provide theoretical evidence for the performance dependence on p of the gradient dynamical systems based on the three new families of NCP functions. This was not accomplished in the preceding chapter.
(vi) This work is a significant improvement of the numerical results that were initially presented in the previous chapter, since the proposed families not only provide faster convergence speeds but also higher stability. That is, the proposed generalizations yield neural networks which are less sensitive to initial conditions, which is one of the main issues encountered in Chapter 4.
In summary, this section can be viewed as an important extension of the works presented in [3, 5, 9, 30] where the discrete-type generalization and two discrete-type symmetrizations of the NR function were studied.