[SciPy-User] Optimization methods for fitting ODEs

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] Optimization methods for fitting ODEs

David Hagen
TL;DR: Anyone have a recommended optimization method for fitting ODEs to data when I have the analytic gradient?

I am currently fitting the numeric solutions of ODEs to data. I wrote an objective function, a gradient on that objective function, and passed both to scipy.optimize.minimize. This worked fine on small problems, but all the scipy minimize methods broke down when the number of parameters got large (~40). My past experience in Matlab suggests that this happens because some parameter directions are MUCH more tightly constrained than others.

The scipy methods are easy-to-use, but I was wondering if there were any alternatives that people had found to work well when the problem got too hard for scipy. Here is a list of features that I probably need:

* black-box optimizer
* local optimizer
* uses supplied analytic gradient
* allows bounds on parameters
* accepts a maximum step size

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Optimization methods for fitting ODEs

ralfgommers


On Wed, Mar 11, 2020 at 2:42 PM David Hagen <[hidden email]> wrote:
TL;DR: Anyone have a recommended optimization method for fitting ODEs to data when I have the analytic gradient?

I am currently fitting the numeric solutions of ODEs to data. I wrote an objective function, a gradient on that objective function, and passed both to scipy.optimize.minimize. This worked fine on small problems, but all the scipy minimize methods broke down when the number of parameters got large (~40). My past experience in Matlab suggests that this happens because some parameter directions are MUCH more tightly constrained than others.

The scipy methods are easy-to-use, but I was wondering if there were any alternatives that people had found to work well when the problem got too hard for scipy. Here is a list of features that I probably need:

* black-box optimizer
* local optimizer
* uses supplied analytic gradient
* allows bounds on parameters
* accepts a maximum step size

I like Emcee (https://emcee.readthedocs.io/en/stable/tutorials/line/) for fitting ODEs to data.

I'm not sure I fully understand what you mean by using the gradient. If you have the analytic gradient, I assume you also have the solution to the ODE, or you can at least numerically integrate the gradient to get that solution. So at that point there's no need to solve the ODE during the fitting process?

Cheers,
Ralf

 
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Optimization methods for fitting ODEs

David Hagen
> I'm not sure I fully understand what you mean by using the gradient.

By analytic gradient, I mean as opposed to a finite difference gradient. You are correct that I still have to numerically integrate the forward sensitivities and then propagate the sensitivities via chain rule through the the objective function.

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user