Mathematica: Evaluation order during numerical optimisation of black box functions

Question

I am attempting to perform a numerical optimisation of a "black box" function in Mathematica. Schematically it goes like this:

NMinimize[{comb[x,y,z], x > 0}, {x,y,z}]

where comb[x,y,z] is defined similarly to this:

comb[x_,y_,z_] := Module[{},
  Print[x,y,z];
  M = FindMaximum[SkewNormal[a,x,y,z], {a,x}] // First;
  val = f[x,y,z,M];
  Return[val];
];

However, all of the minimisation functions I have tried seem to not immediately provide comb[x,y,z] with numerical values, and it ends up trying to evaluate the FindMaximum with symbolic values for x,y,z (which is easily verified because the Print[x,y,z] also evaluates symbolically). The Findmaximum thus fails (FindMaximum::nrnum: The function value blah blah is not a real number) and so the minimisation fails.

How do I fix up the evaluation order so that the sub-functions of comb are evaluated with numerical values?

Knuth

Another (more subtle) problem because of symbolic pre-processing in NMinimize recently was solved in other thread by Daniel Lichtblau. – Hormonal

Hmm ok thanks I'll check that out. I have also just noticed that this problem doesn't occur in Mathematica 8 (I was running 7 previously). – Knuth

Answer 1

Evaluation order for FindMinimum, FindMaximum, FindRoot and FindFit is documented on the tutorial/UnconstrainedOptimizationSymbolicEvaluation Documentation page. I think that something very similar is applicable to the NMinimize function. The description is quite lengthy so I will cite here only the proposed solution from that page:

If your function is such that symbolic evaluation will not keep the function as intended or will be prohibitively slow, you should define your function so that it only evaluates for numerical values of the variables. The simplest way to do this is by defining your function using PatternTest (?), as in f[x_?NumberQ]:=definition.

It may seem that symbolic evaluation just creates a bother since you have to define the function specifically to prevent it. However, without symbolic evaluation, it is hard for Mathematica to take advantage of its unique combination of numerical and symbolic power. Symbolic evaluation means that the commands can consistently take advantage of benefits that come from symbolic analysis, such as algorithm determination, automatic computation of derivatives, automatic optimization and compilation, and structural analysis.

Hormonal

Answer 2

How about changing comb to

comb[x_?NumericQ, y_?NumericQ, z_?NumericQ] := 
 Module[{}, Print[x, y, z];
 M = FindMaximum[SkewNormal[a, x, y, z], {a, x}] // First;
 val = f[x, y, z, M];
 Return[val];];

which causes the definition of comb to be evaluated only if its arguments are numbers?

Immortality

Nah that doesn't help. My function actually did that already, I just removed those bits for the purposes of asking this question. – Knuth

@Ben strange that Print would also show symbolic values, since comb[x,y,z] with symbolic values would not match the definition I gave. Did you Clear[comb] before running this? Maybe it is matching the definition without the NumericQ bit. – Immortality

@Immortality This didn't work for me at first, but after clearing it did the trick. – Indignity