[SciPy-User] Fitting Polynomial With Shape Restrictions

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[SciPy-User] Fitting Polynomial With Shape Restrictions

Jared Vacanti
I am trying to fit a polynomial to observational data with shape restrictions - in this particular case monotonicity (decreasing) of the function and an always positive second derivative.

Some of the interpolation classes have a mathematical "built-in" restriction - like scipy.interpolate.Rbf's thin-plate roughness penalty imposes some restrictions, but it's not explicit or adjustable.

What are my options for imposing boundary conditions or shape restrictions on the spline? 

I have sample data here: 

import pandas as pd

or available independently at https://bpaste.net/raw/3e20878b5237

I have tried using a interior point convex optimization solver, but the results seem to be numerically finicky. Are there other alternatives? 

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fitting Polynomial With Shape Restrictions

josef.pktd


On Sun, May 7, 2017 at 2:45 PM, Jared Vacanti <[hidden email]> wrote:
I am trying to fit a polynomial to observational data with shape restrictions - in this particular case monotonicity (decreasing) of the function and an always positive second derivative.

Some of the interpolation classes have a mathematical "built-in" restriction - like scipy.interpolate.Rbf's thin-plate roughness penalty imposes some restrictions, but it's not explicit or adjustable.

What are my options for imposing boundary conditions or shape restrictions on the spline? 

I have sample data here: 

import pandas as pd

or available independently at https://bpaste.net/raw/3e20878b5237

I have tried using a interior point convex optimization solver, but the results seem to be numerically finicky. Are there other alternatives? 

As far as I know, pchip is the only one with monotonicity constraints  https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.pchip_interpolate.html

Josef

 

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fitting Polynomial With Shape Restrictions

Jared Vacanti
Are there other techniques of fitting splines (perhaps using scipy.optimize ?) where I can impose boundary conditions myself, even outside of the options provided in scipy.interpolate?

On Sun, May 7, 2017 at 2:02 PM, <[hidden email]> wrote:


On Sun, May 7, 2017 at 2:45 PM, Jared Vacanti <[hidden email]> wrote:
I am trying to fit a polynomial to observational data with shape restrictions - in this particular case monotonicity (decreasing) of the function and an always positive second derivative.

Some of the interpolation classes have a mathematical "built-in" restriction - like scipy.interpolate.Rbf's thin-plate roughness penalty imposes some restrictions, but it's not explicit or adjustable.

What are my options for imposing boundary conditions or shape restrictions on the spline? 

I have sample data here: 

import pandas as pd

or available independently at https://bpaste.net/raw/3e20878b5237

I have tried using a interior point convex optimization solver, but the results seem to be numerically finicky. Are there other alternatives? 

As far as I know, pchip is the only one with monotonicity constraints  https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.pchip_interpolate.html

Josef

 

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fitting Polynomial With Shape Restrictions

Evgeni Burovski
On Sun, May 7, 2017 at 10:12 PM, Jared Vacanti <[hidden email]> wrote:
> Are there other techniques of fitting splines (perhaps using scipy.optimize
> ?) where I can impose boundary conditions myself, even outside of the
> options provided in scipy.interpolate?

FITPACK has some spline fitting with convexity constraints, but I
don't think it's exposed to python:

https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack/concon.f
https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack/cocosp.f

Hyman et al have some discussion of constructing local interpolants
with convexity constraints:
http://math.lanl.gov/~mac/papers/numerics/DEH89.pdf
and references therein.



> On Sun, May 7, 2017 at 2:02 PM, <[hidden email]> wrote:
>>
>>
>>
>> On Sun, May 7, 2017 at 2:45 PM, Jared Vacanti <[hidden email]>
>> wrote:
>>>
>>> I am trying to fit a polynomial to observational data with shape
>>> restrictions - in this particular case monotonicity (decreasing) of the
>>> function and an always positive second derivative.
>>>
>>> Some of the interpolation classes have a mathematical "built-in"
>>> restriction - like scipy.interpolate.Rbf's thin-plate roughness penalty
>>> imposes some restrictions, but it's not explicit or adjustable.
>>>
>>> What are my options for imposing boundary conditions or shape
>>> restrictions on the spline?
>>>
>>> I have sample data here:
>>>
>>> import pandas as pd
>>> df = pd.read_csv("https://bpaste.net/raw/3e20878b5237")
>>>
>>> or available independently at https://bpaste.net/raw/3e20878b5237
>>>
>>> I have tried using a interior point convex optimization solver, but the
>>> results seem to be numerically finicky. Are there other alternatives?
>>
>>
>> As far as I know, pchip is the only one with monotonicity constraints
>> https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.pchip_interpolate.html
>>
>> Josef
>>
>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-User mailing list
>>> [hidden email]
>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>
>>
>>
>> _______________________________________________
>> SciPy-User mailing list
>> [hidden email]
>> https://mail.python.org/mailman/listinfo/scipy-user
>>
>
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fitting Polynomial With Shape Restrictions

David Goldsmith
In reply to this post by Jared Vacanti
I'd suggest doing some advance pencil-and-paper work to determine what your constraints imply about the relationships among the coefficients.  For example, you mention that the second derivative should always be positive: you can readily formalize this by writing down the second derative of your model, adding a slack variable, which you constrain to be strictly negative, and then set that second-derivative-plus-slack expression equal to zero: oui la, an additional equation--which can be added to the matrix representation of your minimization problem--to rigorously represent your second derivative constraint!  The monotonicity requirement can be treated similarly, as a limitation on the first derivative, though it may require introduction of additional parameters, perhaps in the form of bounds on the interval over which you plan to use the model.  But in any event, with a little pre-programming effort, you should be able to readily solve your problem without having to resort to any programming "tricks" or "workarounds."  (I was once charged with writing a program to explore the nine-dimensional parameter space of eighth-order polynomials to look for sub-spaces with particular properties; too late was i informed that there were additional "pre-constraints," e.g., monotonicity such as you describe, and strict positiveness of the polynomial output--that i could have used to significantly reduce both the dimensionality of and the ranges in the space I needed to search.  This was in the area of plasma confinement simulation: may i ask, in what area you are working?)

HTH,

DLG

On Sun, May 7, 2017 at 12:13 PM Jared Vacanti <[hidden email]> wrote:
Are there other techniques of fitting splines (perhaps using scipy.optimize ?) where I can impose boundary conditions myself, even outside of the options provided in scipy.interpolate?

On Sun, May 7, 2017 at 2:02 PM, <[hidden email]> wrote:


On Sun, May 7, 2017 at 2:45 PM, Jared Vacanti <[hidden email]> wrote:
I am trying to fit a polynomial to observational data with shape restrictions - in this particular case monotonicity (decreasing) of the function and an always positive second derivative.

Some of the interpolation classes have a mathematical "built-in" restriction - like scipy.interpolate.Rbf's thin-plate roughness penalty imposes some restrictions, but it's not explicit or adjustable.

What are my options for imposing boundary conditions or shape restrictions on the spline? 

I have sample data here: 

import pandas as pd

or available independently at https://bpaste.net/raw/3e20878b5237

I have tried using a interior point convex optimization solver, but the results seem to be numerically finicky. Are there other alternatives? 

As far as I know, pchip is the only one with monotonicity constraints  https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.pchip_interpolate.html

Josef

 

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fitting Polynomial With Shape Restrictions

josef.pktd
In reply to this post by Evgeni Burovski


On Sun, May 7, 2017 at 3:34 PM, Evgeni Burovski <[hidden email]> wrote:
On Sun, May 7, 2017 at 10:12 PM, Jared Vacanti <[hidden email]> wrote:
> Are there other techniques of fitting splines (perhaps using scipy.optimize
> ?) where I can impose boundary conditions myself, even outside of the
> options provided in scipy.interpolate?

FITPACK has some spline fitting with convexity constraints, but I
don't think it's exposed to python:

https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack/concon.f
https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack/cocosp.f

Hyman et al have some discussion of constructing local interpolants
with convexity constraints:
http://math.lanl.gov/~mac/papers/numerics/DEH89.pdf
and references therein.


I looked into monotonicity and similar constraints for splines several years ago.
I gave up because adding all the constraints that are implied on the coefficients (values and derivatives at the knots) looked difficult, and it is difficult to get the relevant parameters directly from the scipy splines.

AFAIR
and I was hoping for somebody to come along with more background and more incentive than I had.

Josef

 


> On Sun, May 7, 2017 at 2:02 PM, <[hidden email]> wrote:
>>
>>
>>
>> On Sun, May 7, 2017 at 2:45 PM, Jared Vacanti <[hidden email]>
>> wrote:
>>>
>>> I am trying to fit a polynomial to observational data with shape
>>> restrictions - in this particular case monotonicity (decreasing) of the
>>> function and an always positive second derivative.
>>>
>>> Some of the interpolation classes have a mathematical "built-in"
>>> restriction - like scipy.interpolate.Rbf's thin-plate roughness penalty
>>> imposes some restrictions, but it's not explicit or adjustable.
>>>
>>> What are my options for imposing boundary conditions or shape
>>> restrictions on the spline?
>>>
>>> I have sample data here:
>>>
>>> import pandas as pd
>>> df = pd.read_csv("https://bpaste.net/raw/3e20878b5237")
>>>
>>> or available independently at https://bpaste.net/raw/3e20878b5237
>>>
>>> I have tried using a interior point convex optimization solver, but the
>>> results seem to be numerically finicky. Are there other alternatives?
>>
>>
>> As far as I know, pchip is the only one with monotonicity constraints
>> https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.pchip_interpolate.html
>>
>> Josef
>>
>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-User mailing list
>>> [hidden email]
>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>
>>
>>
>> _______________________________________________
>> SciPy-User mailing list
>> [hidden email]
>> https://mail.python.org/mailman/listinfo/scipy-user
>>
>
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fitting Polynomial With Shape Restrictions

David Goldsmith
Hi, josef.  If you mean having generic code that automates the process, then you are probably correct: there's just too many degrees of freedom to account for all possibilities in a generic program (at least that's the way it appears to me, too).  But using known math to work out, "by hand," these implications in specific cases prior to using the generic code tools we do have, largely thanks to you, is eminently "do-able."  Thanks for all you've done for scipy, lo these many years!

DLG

On Sun, May 7, 2017 at 1:37 PM <[hidden email]> wrote:
On Sun, May 7, 2017 at 3:34 PM, Evgeni Burovski <[hidden email]> wrote:
On Sun, May 7, 2017 at 10:12 PM, Jared Vacanti <[hidden email]> wrote:
> Are there other techniques of fitting splines (perhaps using scipy.optimize
> ?) where I can impose boundary conditions myself, even outside of the
> options provided in scipy.interpolate?

FITPACK has some spline fitting with convexity constraints, but I
don't think it's exposed to python:

https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack/concon.f
https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack/cocosp.f

Hyman et al have some discussion of constructing local interpolants
with convexity constraints:
http://math.lanl.gov/~mac/papers/numerics/DEH89.pdf
and references therein.


I looked into monotonicity and similar constraints for splines several years ago.
I gave up because adding all the constraints that are implied on the coefficients (values and derivatives at the knots) looked difficult, and it is difficult to get the relevant parameters directly from the scipy splines.

AFAIR
and I was hoping for somebody to come along with more background and more incentive than I had.

Josef

 


> On Sun, May 7, 2017 at 2:02 PM, <[hidden email]> wrote:
>>
>>
>>
>> On Sun, May 7, 2017 at 2:45 PM, Jared Vacanti <[hidden email]>
>> wrote:
>>>
>>> I am trying to fit a polynomial to observational data with shape
>>> restrictions - in this particular case monotonicity (decreasing) of the
>>> function and an always positive second derivative.
>>>
>>> Some of the interpolation classes have a mathematical "built-in"
>>> restriction - like scipy.interpolate.Rbf's thin-plate roughness penalty
>>> imposes some restrictions, but it's not explicit or adjustable.
>>>
>>> What are my options for imposing boundary conditions or shape
>>> restrictions on the spline?
>>>
>>> I have sample data here:
>>>
>>> import pandas as pd
>>> df = pd.read_csv("https://bpaste.net/raw/3e20878b5237")
>>>
>>> or available independently at https://bpaste.net/raw/3e20878b5237
>>>
>>> I have tried using a interior point convex optimization solver, but the
>>> results seem to be numerically finicky. Are there other alternatives?
>>
>>
>> As far as I know, pchip is the only one with monotonicity constraints
>> https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.pchip_interpolate.html
>>
>> Josef
>>
>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-User mailing list
>>> [hidden email]
>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>
>>
>>
>> _______________________________________________
>> SciPy-User mailing list
>> [hidden email]
>> https://mail.python.org/mailman/listinfo/scipy-user
>>
>
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Loading...