It would be very helpful if one could specify `ftol` and `xtol` with any of the optimization algorithms. How difficult would it be to implement this?
Phillip 
It does exist for fmin http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin.html#scipy.optimize.fmin Which one do you want to use?  2013/6/5 pfeldman <[hidden email]> It would be very helpful if one could specify `ftol` and `xtol` with any of _______________________________________________ SciPyUser mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipyuser 
In particular, I'd like to be able to specify `ftol` and `xtol` with optimize.fmin_bfgs, optimize.fmin_l_bfgs_b, and optimize.anneal. optimize.anneal has a parameter called `feps` that seems similar to `ftol`, but there is no parameter comparable to `xtol`. Also, it would be great if the parameter names were the same across the boardto the extent possiblebecause that would make it much easier to compare alternative optimization algorithms.

I believe that making the optimization interfaces more uniform would be a substantial improvement, and I'm somewhat disappointed that there has been no discussion on this topic.

On Mon, Jun 24, 2013 at 10:52 PM, pfeldman <[hidden email]> wrote: I believe that making the optimization interfaces more uniform would be a Scipy 0.11 made a major step in unifying the interfaces: http://docs.scipy.org/doc/scipydev/reference/release.0.11.0.html#scipyoptimizeimprovements That was extensively discussed. More (backwardscompatible) improvements in this direction are of course very welcome. Ralf _______________________________________________ SciPyUser mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipyuser 
I am using the latest released version of SciPy, and I agree that the interfaces are much improved. The termination conditions for the optimizers is an area where the interfaces are still far from uniform. To allow termination conditions to be specified in a uniform fashion without breaking backwards compatibility, one would probably have to support two interfaces.
Phillip On Wed, Jun 26, 2013 at 2:05 PM, Ralf Gommers3 [via ScipyUser] <[hidden email]> wrote:

On the subject of ftol and xtol, the help for fmin describes these parameters as :
xtol : float Relative error in xopt acceptable for convergence. ftol : number Relative error in func(xopt) acceptable for convergence. The use of the word "Relative" here has always implied to me that the convergence tests calculated by fmin refer to fractional changes in parameters or function values. However, I had difficulties trying to minimize a function in which the two parameters defer by many orders or magnitude (e.g. 1 and 1e10) and I've come to the conclusion it is because the large parameter never passes the convergence test. The code in fmin which tests for convergence is : if (max(numpy.ravel(abs(sim[1:]sim[0]))) <= xtol \ and max(abs(fsim[0]fsim[1:])) <= ftol): break This does not look like a test for the relative change in parameters or function values to me, but rather a test for their absolute changes. A relative change would surely be something like if (max(numpy.ravel(abs((sim[1:]sim[0])/sim[0]))) <= xtol \ and max(abs((fsim[0]fsim[1:])/fsim[0])) <= ftol): break (though care should really be taken for possible divide by zero errors). Either the docstring needs changing so "Relative" is replaced with "Absolute" to make it clear, or the convergence test needs to be changed so it truly is a relative error test. 
If one uses relative error, it is unclear how to handle the situation where the minimum is at zero. So, absolute error seems more practical. On the subject of ftol and xtol, the help for fmin describes these parameters as : 
Free forum by Nabble  Edit this page 