Non-linear non-constrained minimization of a function with known gradient

defined on an n-dimensional Euclidean space, using the Nonlinear Conjugate Gradient method. The implementation was done based on the beautifully clear explanatory article [1].

The method can be seen as an adaptation of a standard Conjugate Gradient method for numerically solving the systems of linear equations.

It should be noted, that this method, although deterministic, is rather a heuristic method and therefore may converge to a local minima, not necessary a global one. What is even more disastrous, most of its behaviour is ruled by gradient, therefore it essentially cannot distinguish between local minima and maxima. Therefore, if it starts sufficiently near to the local maximum, it may converge to it. Another obvious restriction is that it should be possible to compute the gradient of a function at any point, thus it is preferable to have analytic expression for gradient and computational burden should be born by the user.

The latter responsibility is accompilished via the getGradient method of the function being optimized). This method takes point a point in n-dimensional space (input argument represents the array of coordinates of that point) and comput its gradient (it should be returned in the output as an array).

NOTE: term criteria should meet one of the following conditions:

• type == 'Count+EPS' && epsilon > 0 && maxCount > 0
• type == 'Count' && maxCount > 0

References

[1]:

An Introduction to the Conjugate Gradient Method Without the Agonizing Pain PDF by Jonathan Richard Shewchuk.