Problem using optimize.fmin_slsqp

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem using optimize.fmin_slsqp

Paweł Kwaśniewski
Hello,

I need to fit some data using a constrained least square fit - unconstrained fit gives me a good 'visual' fit, but the parameters are non-physical, therefore useless. I found that optimize.fmin_slsqp is what I want to use. I tried it, but I'm stuck with some error I completely don't understand... I know how to use the minimization function - I played with it a bit on simulated data, and it works well. I think the problem might be with my fitting function - it's quite lengthy, probably resource consuming. But maybe it's something else. Anyway, here's what I'm doing:

    params, fval, its, imode, smode = optimize.fmin_slsqp(residuals, guess,
                                             args = (points,vals,errs),
                                             bounds = b,
                                             full_output = True)

residuals is a function which returns a float, being the sum of squared residuals (just didn't change the name after using non-linear least square fit). What I'm getting is:

Inequality constraints incompatible    (Exit mode 4)
            Current function value: 2.18747774338
            Iterations: 1
            Function evaluations: 7
            Gradient evaluations: 1
*** glibc detected *** python: double free or corruption (!prev): 0x08d465f0 ***

As you can see in the function call, I'm not even using inequality constraints, so I don't understand why it complains about it. The last line is a mystery for me. It seems that after one iteration something goes really wrong... I would really appreciate some advice on how can I debug my code. Please tell me what else I should provide to be more clear.

Regards,

Pawel

--
Pawel

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Problem using optimize.fmin_slsqp

jseabold
2010/11/26 Paweł Kwaśniewski <[hidden email]>:

> Hello,
>
> I need to fit some data using a constrained least square fit - unconstrained
> fit gives me a good 'visual' fit, but the parameters are non-physical,
> therefore useless. I found that optimize.fmin_slsqp is what I want to use. I
> tried it, but I'm stuck with some error I completely don't understand... I
> know how to use the minimization function - I played with it a bit on
> simulated data, and it works well. I think the problem might be with my
> fitting function - it's quite lengthy, probably resource consuming. But
> maybe it's something else. Anyway, here's what I'm doing:
>
>     params, fval, its, imode, smode = optimize.fmin_slsqp(residuals, guess,
>                                              args = (points,vals,errs),
>                                              bounds = b,
>                                              full_output = True)
>
> residuals is a function which returns a float, being the sum of squared
> residuals (just didn't change the name after using non-linear least square
> fit). What I'm getting is:
>
> Inequality constraints incompatible    (Exit mode 4)
>             Current function value: 2.18747774338
>             Iterations: 1
>             Function evaluations: 7
>             Gradient evaluations: 1
> *** glibc detected *** python: double free or corruption (!prev): 0x08d465f0
> ***
>
> As you can see in the function call, I'm not even using inequality
> constraints, so I don't understand why it complains about it. The last line
> is a mystery for me. It seems that after one iteration something goes really
> wrong... I would really appreciate some advice on how can I debug my code.
> Please tell me what else I should provide to be more clear.
>

Can you provide some code and data to replicate the problem?

Skipper
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Problem using optimize.fmin_slsqp

Didrik Pinte-5
In reply to this post by Paweł Kwaśniewski
2010/11/26 Paweł Kwaśniewski <[hidden email]>:

> Hello,
>
> I need to fit some data using a constrained least square fit - unconstrained
> fit gives me a good 'visual' fit, but the parameters are non-physical,
> therefore useless. I found that optimize.fmin_slsqp is what I want to use. I
> tried it, but I'm stuck with some error I completely don't understand... I
> know how to use the minimization function - I played with it a bit on
> simulated data, and it works well. I think the problem might be with my
> fitting function - it's quite lengthy, probably resource consuming. But
> maybe it's something else. Anyway, here's what I'm doing:
>
>     params, fval, its, imode, smode = optimize.fmin_slsqp(residuals, guess,
>                                              args = (points,vals,errs),
>                                              bounds = b,
>                                              full_output = True)
>
> residuals is a function which returns a float, being the sum of squared
> residuals (just didn't change the name after using non-linear least square
> fit). What I'm getting is:
>
> Inequality constraints incompatible    (Exit mode 4)
>             Current function value: 2.18747774338
>             Iterations: 1
>             Function evaluations: 7
>             Gradient evaluations: 1
> *** glibc detected *** python: double free or corruption (!prev): 0x08d465f0
> ***
>
> As you can see in the function call, I'm not even using inequality
> constraints, so I don't understand why it complains about it.

You do at least have bounds which should be represented like
inequality contraints. I don't know about the internals of the
algorithms but that might be the problem.

-- Didrik
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Problem using optimize.fmin_slsqp

basweber
In reply to this post by Paweł Kwaśniewski
Paweł Kwaśniewski wrote:

> Hello,
>
> I need to fit some data using a constrained least square fit -
> unconstrained fit gives me a good 'visual' fit, but the parameters are
> non-physical, therefore useless. I found that optimize.fmin_slsqp is
> what I want to use. I tried it, but I'm stuck with some error I
> completely don't understand... I know how to use the minimization
> function - I played with it a bit on simulated data, and it works well.
> I think the problem might be with my fitting function - it's quite
> lengthy, probably resource consuming. But maybe it's something else.
> Anyway, here's what I'm doing:
>
>     params, fval, its, imode, smode = optimize.fmin_slsqp(residuals, guess,
>                                              args = (points,vals,errs),
>                                              bounds = b,
>                                              full_output = True)
>
> residuals is a function which returns a float, being the sum of squared
> residuals (just didn't change the name after using non-linear least
> square fit). What I'm getting is:
>
> Inequality constraints incompatible    (Exit mode 4)
>             Current function value: 2.18747774338
>             Iterations: 1
>             Function evaluations: 7
>             Gradient evaluations: 1
> *** glibc detected *** python: double free or corruption (!prev):
> 0x08d465f0 ***
>


Hello Paweł,

maybe it helps to increase the verbosity of fmin_slsqp using the
additional argument iprint=2.

Another debugging strategy would be, to let your function 'residuals'
print everything it gets as argument and its return value. This way you
could determine on which data the error happens.


Regards,
Bastian
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Problem using optimize.fmin_slsqp

Paweł Kwaśniewski
Hi,

Sorry, I didn't have time during the weekend to sit and write this down, but I tried several things and I think I'm a bit closer to solving my problem.

Skipper, how should I provide the code and data? I don't want to put everything into e-mail text and flood everyone with my (not so nice) code. Is it OK to send an attached zip with the files?

So, I managed to avoid the error I had before by giving additional input to optimize.fmin_slsqp, namely - providing epsilon for the approximation of the Jacobian matrix and/or the approximation of first derivative of the function. I'm not sure how to choose an optimal value for it, so I just tried several, with different results. Later, I checked what is the default value for epsilon in this function (I found the source code for fmin_slsqp and printed this and that...) - it's very small: 1.49e-8. I was trying something in the range of 1 or 10. Now if I left the default value or gave for example 1e-3, I got this:

epsilon = 1.00e-03
  NIT    FC           OBJFUN            GNORM
python: malloc.c:4631: _int_malloc: Assertion `(unsigned long)(size) >= (unsigned long)(nb)' failed.

So I got a different error, clearly something with memory allocation, if I understand well. I suspect the problem may be caused by the fact that the domain of my fitting function is in the range of 1e8. Giving very small epsilon (used as dx to calculate the derivative) may raise some kind of problems related to finite precision of the machine. Since the Jacobian is not calculated during my fit (I put a print statement in the approx_jacobian function, the file is in /usr/lib/python2.6/site-packages/scipy/optimize/slsqp.py - no output from there), it's the approx_fprime() which makes the problem (defined in /usr/lib/python2.6/site-packages/scipy/optimize/optimize.py - checked that with print -it works).

My conclusion is that it's a bug (unhandled exception I guess) of the approx_fprime() function. I also noticed, that when I give the error generating value of epsilon and run the code for the first time with it, it gives the >*** glibc detected *** python: double free or corruption (!prev): 0x08df6240 *** error. If I try again to run the code with the same values it spits out:
>python: malloc.c:4631: _int_malloc: Assertion `(unsigned long)(size) >= (unsigned long)(nb)' failed.
Aborted
This behavior I don't understand.

Regards,

Pawel





2010/11/28 Bastian Weber <[hidden email]>
Paweł Kwaśniewski wrote:
> Hello,
>
> I need to fit some data using a constrained least square fit -
> unconstrained fit gives me a good 'visual' fit, but the parameters are
> non-physical, therefore useless. I found that optimize.fmin_slsqp is
> what I want to use. I tried it, but I'm stuck with some error I
> completely don't understand... I know how to use the minimization
> function - I played with it a bit on simulated data, and it works well.
> I think the problem might be with my fitting function - it's quite
> lengthy, probably resource consuming. But maybe it's something else.
> Anyway, here's what I'm doing:
>
>     params, fval, its, imode, smode = optimize.fmin_slsqp(residuals, guess,
>                                              args = (points,vals,errs),
>                                              bounds = b,
>                                              full_output = True)
>
> residuals is a function which returns a float, being the sum of squared
> residuals (just didn't change the name after using non-linear least
> square fit). What I'm getting is:
>
> Inequality constraints incompatible    (Exit mode 4)
>             Current function value: 2.18747774338
>             Iterations: 1
>             Function evaluations: 7
>             Gradient evaluations: 1
> *** glibc detected *** python: double free or corruption (!prev):
> 0x08d465f0 ***
>


Hello Paweł,

maybe it helps to increase the verbosity of fmin_slsqp using the
additional argument iprint=2.

Another debugging strategy would be, to let your function 'residuals'
print everything it gets as argument and its return value. This way you
could determine on which data the error happens.


Regards,
Bastian
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user



--
Paweł

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Problem using optimize.fmin_slsqp

jseabold
2010/11/29 Paweł Kwaśniewski <[hidden email]>:
> Hi,
>
> Sorry, I didn't have time during the weekend to sit and write this down, but
> I tried several things and I think I'm a bit closer to solving my problem.
>
> Skipper, how should I provide the code and data? I don't want to put
> everything into e-mail text and flood everyone with my (not so nice) code.
> Is it OK to send an attached zip with the files?

How long is your code?  Can you boil it down to a small example that
replicates the problem?  With a subsample of the data or a random
sample?

>
> So, I managed to avoid the error I had before by giving additional input to
> optimize.fmin_slsqp, namely - providing epsilon for the approximation of the
> Jacobian matrix and/or the approximation of first derivative of the
> function. I'm not sure how to choose an optimal value for it, so I just
> tried several, with different results. Later, I checked what is the default
> value for epsilon in this function (I found the source code for fmin_slsqp
> and printed this and that...) - it's very small: 1.49e-8. I was trying
> something in the range of 1 or 10. Now if I left the default value or gave
> for example 1e-3, I got this:
>
> epsilon = 1.00e-03
>   NIT    FC           OBJFUN            GNORM
> python: malloc.c:4631: _int_malloc: Assertion `(unsigned long)(size) >=
> (unsigned long)(nb)' failed.
>
> So I got a different error, clearly something with memory allocation, if I
> understand well. I suspect the problem may be caused by the fact that the
> domain of my fitting function is in the range of 1e8. Giving very small
> epsilon (used as dx to calculate the derivative) may raise some kind of
> problems related to finite precision of the machine. Since the Jacobian is
> not calculated during my fit (I put a print statement in the approx_jacobian
> function, the file is in
> /usr/lib/python2.6/site-packages/scipy/optimize/slsqp.py - no output from
> there), it's the approx_fprime() which makes the problem (defined in
> /usr/lib/python2.6/site-packages/scipy/optimize/optimize.py - checked that
> with print -it works).
>
> My conclusion is that it's a bug (unhandled exception I guess) of the
> approx_fprime() function. I also noticed, that when I give the error
> generating value of epsilon and run the code for the first time with it, it
> gives the >*** glibc detected *** python: double free or corruption (!prev):
> 0x08df6240 *** error. If I try again to run the code with the same values it
> spits out:
>>python: malloc.c:4631: _int_malloc: Assertion `(unsigned long)(size) >=
>> (unsigned long)(nb)' failed.
> Aborted
> This behavior I don't understand.
>

I can't see why the approx_fprime itself should cause a seg fault.  As
you mentioned, it could be a scaling issue (?), but regardless I don't
think this should happen.

Skipper
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Problem using optimize.fmin_slsqp

jseabold
On Tue, Nov 30, 2010 at 11:01 AM, Skipper Seabold <[hidden email]> wrote:

> 2010/11/29 Paweł Kwaśniewski <[hidden email]>:
>> Hi,
>>
>> Sorry, I didn't have time during the weekend to sit and write this down, but
>> I tried several things and I think I'm a bit closer to solving my problem.
>>
>> Skipper, how should I provide the code and data? I don't want to put
>> everything into e-mail text and flood everyone with my (not so nice) code.
>> Is it OK to send an attached zip with the files?
>
> How long is your code?  Can you boil it down to a small example that
> replicates the problem?  With a subsample of the data or a random
> sample?
>

This looks like it could be the same issue:
http://projects.scipy.org/scipy/ticket/1333

Can you confirm?

Skipper
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user