What algorithm to use for optimization af a function of N variables?

Question

Given the function y=f(x1,x2,x3,x4,x5,x6), I'm trying to find the values of x1..x6, where the function reaches its maximum. The function is quite well behaved, continuous, with a single peak with the value of 1 and low-level, noise-like values around it. The function is quite costly to evaluate, so I'm looking to limit evaluations to a minimum, which means silly ways of iteratively searching for the peak like golden section for each dimension separately are out of the question - I need something which converges quickly to the maximum.

I'm aware of the existence of various algorithms like the Levenberg-Marquardt algorithm, the simplex method in multi-dimensions, the Powell method etc. The problem is that I don't know which one would be best, and at the same time - relatively simple, because honestly looking at the theory behind them gives me the creeps.

Perhaps there is a library for function optimization that I could use? I'm aware of MINPACK which implements L-M among others, but from what I can tell, it's used for systems of M equations of N variables where M >= N, and I have only one equation with 6 free variables.

Answer 1

Ok so basically you have a multivariate objective function and you wish to perform unconstrained maximisation on it. You also mentioned that you might have some noise at places and that function evaluations are expensive.

Based on this I would suggest "Subplex" which you can find in the opt section of Netlib. Its a generalisation of Nelder-Mead and requires relatively few function evaluations (linear with problem size). Its also good for noisy objective functions.

Answer 2

I found a website providing a decision mechanism for determining the best approach to optimizing a particular case:

http://plato.asu.edu/guide.html