ftol and xtol

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

ftol and xtol

pfeldman
It would be very helpful if one could specify `ftol` and `xtol` with any of the optimization algorithms.  How difficult would it be to implement this?

Phillip
Reply | Threaded
Open this post in threaded view
|

Re: ftol and xtol

Oleksandr Huziy
Which one do you want to use?

--
Oleksandr (Sasha) Huziy


2013/6/5 pfeldman <[hidden email]>
It would be very helpful if one could specify `ftol` and `xtol` with any of
the optimization algorithms.  How difficult would it be to implement this?

Phillip



--
View this message in context: http://scipy-user.10969.n7.nabble.com/ftol-and-xtol-tp18355.html
Sent from the Scipy-User mailing list archive at Nabble.com.
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: ftol and xtol

pfeldman
In particular, I'd like to be able to specify `ftol` and `xtol` with optimize.fmin_bfgs, optimize.fmin_l_bfgs_b, and optimize.anneal.  optimize.anneal has a parameter called `feps` that seems similar to `ftol`, but there is no parameter comparable to `xtol`.  Also, it would be great if the parameter names were the same across the board--to the extent possible--because that would make it much easier to compare alternative optimization algorithms.
Reply | Threaded
Open this post in threaded view
|

Re: ftol and xtol

pfeldman
I believe that making the optimization interfaces more uniform would be a substantial improvement, and I'm somewhat disappointed that there has been no discussion on this topic.
Reply | Threaded
Open this post in threaded view
|

Re: ftol and xtol

ralfgommers



On Mon, Jun 24, 2013 at 10:52 PM, pfeldman <[hidden email]> wrote:
I believe that making the optimization interfaces more uniform would be a
substantial improvement, and I'm somewhat disappointed that there has been
no discussion on this topic.

That was extensively discussed. More (backwards-compatible) improvements in this direction are of course very welcome.

Ralf
 


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: ftol and xtol

pfeldman
I am using the latest released version of SciPy, and I agree that the interfaces are much improved.  The termination conditions for the optimizers is an area where the interfaces are still far from uniform.  To allow termination conditions to be specified in a uniform fashion without breaking backwards compatibility, one would probably have to support two interfaces.

Phillip

On Wed, Jun 26, 2013 at 2:05 PM, Ralf Gommers-3 [via Scipy-User] <[hidden email]> wrote:



On Mon, Jun 24, 2013 at 10:52 PM, pfeldman <[hidden email]> wrote:
I believe that making the optimization interfaces more uniform would be a
substantial improvement, and I'm somewhat disappointed that there has been
no discussion on this topic.

That was extensively discussed. More (backwards-compatible) improvements in this direction are of course very welcome.

Ralf
 


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user



If you reply to this email, your message will be added to the discussion below:
http://scipy-user.10969.n7.nabble.com/ftol-and-xtol-tp18355p18466.html
To unsubscribe from ftol and xtol, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: ftol and xtol

GRBChaser
On the subject of ftol and xtol,  the  help for fmin describes these parameters as :

    xtol : float
        Relative error in xopt acceptable for convergence.
    ftol : number
        Relative error in func(xopt) acceptable for convergence.


The use of the word "Relative" here has always implied to me that the convergence tests calculated by fmin refer to fractional changes in parameters or function values. However, I had difficulties trying to minimize a function in which the two parameters defer by many orders or magnitude (e.g. 1 and 1e10) and I've come to the conclusion it is because the large parameter never passes the convergence test.

The code in fmin which tests for convergence is :

        if (max(numpy.ravel(abs(sim[1:]-sim[0]))) <= xtol \
            and max(abs(fsim[0]-fsim[1:])) <= ftol):
            break

This does not look like a test for the relative change in parameters or function values to me, but rather a test for their absolute changes. A relative change would surely be something like

        if (max(numpy.ravel(abs((sim[1:]-sim[0])/sim[0]))) <= xtol \
            and max(abs((fsim[0]-fsim[1:])/fsim[0])) <= ftol):
            break

(though care should really be taken for possible divide by zero errors).


Either the docstring needs changing so "Relative" is replaced with "Absolute" to make it clear, or the convergence test needs to be changed so it truly is a relative error test.

Reply | Threaded
Open this post in threaded view
|

Re: ftol and xtol

pfeldman
If one uses relative error, it is unclear how to handle the situation where the minimum is at zero.  So, absolute error seems more practical.

On Fri, Jul 5, 2013 at 4:38 AM, GRBChaser [via Scipy-User] <[hidden email]> wrote:
On the subject of ftol and xtol,  the  help for fmin describes these parameters as :

    xtol : float
        Relative error in xopt acceptable for convergence.
    ftol : number
        Relative error in func(xopt) acceptable for convergence.


The use of the word "Relative" here has always implied to me that the convergence tests calculated by fmin refer to fractional changes in parameters or function values. However, I had difficulties trying to minimize a function in which the two parameters defer by many orders or magnitude (e.g. 1 and 1e10) and I've come to the conclusion it is because the large parameter never passes the convergence test.

The code in fmin which tests for convergence is :

        if (max(numpy.ravel(abs(sim[1:]-sim[0]))) <= xtol \
            and max(abs(fsim[0]-fsim[1:])) <= ftol):
            break

This does not look like a test for the relative change in parameters or function values to me, but rather a test for their absolute changes. A relative change would surely be something like

        if (max(numpy.ravel(abs((sim[1:]-sim[0])/sim[0]))) <= xtol \
            and max(abs((fsim[0]-fsim[1:])/fsim[0])) <= ftol):
            break

(though care should really be taken for possible divide by zero errors).


Either the docstring needs changing so "Relative" is replaced with "Absolute" to make it clear, or the convergence test needs to be changed so it truly is a relative error test.




If you reply to this email, your message will be added to the discussion below:
http://scipy-user.10969.n7.nabble.com/ftol-and-xtol-tp18355p18518.html
To unsubscribe from ftol and xtol, click here.
NAML