[SciPy-User] Multivariate linear (bilinear) fit

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] Multivariate linear (bilinear) fit

Daniele Nicolodi
Hello,

before I write my own, does SciPy come with a function for multivariate
(often also called bilinear, in the case of two variables) linear fits?

I would like to fit a function if the form:

  y = c + p1 * x1 + p2 * x2 + ... + pn * xn

which is a simple least squares problem, but I would like to avoid to
write the code :-)

Thanks. Cheers,
Daniele

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Multivariate linear (bilinear) fit

Paweł Kwaśniewski
Hi Daniele,

Do you know lmfit (https://lmfit.github.io/lmfit-py/)? It's really simple to use and very well documented. 

Cheers,

Pawel

Paweł

2017-04-18 19:17 GMT+02:00 Daniele Nicolodi <[hidden email]>:
Hello,

before I write my own, does SciPy come with a function for multivariate
(often also called bilinear, in the case of two variables) linear fits?

I would like to fit a function if the form:

  y = c + p1 * x1 + p2 * x2 + ... + pn * xn

which is a simple least squares problem, but I would like to avoid to
write the code :-)

Thanks. Cheers,
Daniele

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Multivariate linear (bilinear) fit

Sergio Rojas

----------------------------------------------------------------------

Message: 1
Date: Tue, 18 Apr 2017 11:17:26 -0600
From: Daniele Nicolodi <[hidden email]>
To: [hidden email]
Subject: [SciPy-User] Multivariate linear (bilinear) fit
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=utf-8

Hello,

before I write my own, does SciPy come with a function for multivariate
(often also called bilinear, in the case of two variables) linear fits?

I would like to fit a function if the form:

y = c + p1 * x1 + p2 * x2 + ... + pn * xn

which is a simple least squares problem, but I would like to avoid to
write the code :-)

Thanks. Cheers,
Daniele

=====================

Take a look at the optimization tools from SciPy

    http://www.scipy-lectures.org/advanced/mathematical_optimization/index.html

Sergio

Enhance your #MachineLearning and #BigData skills via #Python #SciPy:

1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video
2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Multivariate linear (bilinear) fit

Nathaniel Smith
In reply to this post by Daniele Nicolodi
On Tue, Apr 18, 2017 at 10:17 AM, Daniele Nicolodi <[hidden email]> wrote:

> Hello,
>
> before I write my own, does SciPy come with a function for multivariate
> (often also called bilinear, in the case of two variables) linear fits?
>
> I would like to fit a function if the form:
>
>   y = c + p1 * x1 + p2 * x2 + ... + pn * xn
>
> which is a simple least squares problem, but I would like to avoid to
> write the code :-)

np.linalg.lstsq and scipy.linalg.lstsq handle both univariate and
multivariate problems. They're pretty bare-bones, though, and they
also compute the sum of squared residuals which can make them slow if
you don't need it and are solving lots of problems in a tight loop.

-n

--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Multivariate linear (bilinear) fit

Joe Kington
They're relatively recent additions, but numpy.polynomial.polyvander2d and numpy.polynomial.polyval2d should also do what you want, unless I'm misunderstanding the problem.


You can also do things like (you could generalize this to N-dimensions, as well):

def polyfit2d(x, y, z, order=3):
    ncols = (order + 1)**2
    G = np.zeros((x.size, ncols))
    ij = itertools.product(range(order+1), range(order+1))
    for k, (i,j) in enumerate(ij):
        G[:,k] = x**i * y**j
    m, _, _, _ = np.linalg.lstsq(G, z)
    return m

def polyval2d(x, y, m):
    order = int(np.sqrt(len(m))) - 1
    ij = itertools.product(range(order+1), range(order+1))
    z = np.zeros_like(x)
    for a, (i,j) in zip(m, ij):
        z += a * x**i * y**j
    return z

On Wed, Apr 19, 2017 at 4:17 PM, Nathaniel Smith <[hidden email]> wrote:
On Tue, Apr 18, 2017 at 10:17 AM, Daniele Nicolodi <[hidden email]> wrote:
> Hello,
>
> before I write my own, does SciPy come with a function for multivariate
> (often also called bilinear, in the case of two variables) linear fits?
>
> I would like to fit a function if the form:
>
>   y = c + p1 * x1 + p2 * x2 + ... + pn * xn
>
> which is a simple least squares problem, but I would like to avoid to
> write the code :-)

np.linalg.lstsq and scipy.linalg.lstsq handle both univariate and
multivariate problems. They're pretty bare-bones, though, and they
also compute the sum of squared residuals which can make them slow if
you don't need it and are solving lots of problems in a tight loop.

-n

--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Multivariate linear (bilinear) fit

Charles R Harris


On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington <[hidden email]> wrote:
They're relatively recent additions, but numpy.polynomial.polyvander2d and numpy.polynomial.polyval2d should also do what you want, unless I'm misunderstanding the problem.


You can also do things like (you could generalize this to N-dimensions, as well):

def polyfit2d(x, y, z, order=3):
    ncols = (order + 1)**2
    G = np.zeros((x.size, ncols))
    ij = itertools.product(range(order+1), range(order+1))
    for k, (i,j) in enumerate(ij):
        G[:,k] = x**i * y**j
    m, _, _, _ = np.linalg.lstsq(G, z)
    return m

def polyval2d(x, y, m):
    order = int(np.sqrt(len(m))) - 1
    ij = itertools.product(range(order+1), range(order+1))
    z = np.zeros_like(x)
    for a, (i,j) in zip(m, ij):
        z += a * x**i * y**j
    return z


I think the "bilinear" is a mistake, as bilinear usually means terms of degree two. AFAICT, this question is just about multivariate linear fits only

<snip>

Chuck




_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Multivariate linear (bilinear) fit

josef.pktd
On Wed, Apr 19, 2017 at 8:12 PM, Charles R Harris
<[hidden email]> wrote:

>
>
> On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington <[hidden email]> wrote:
>>
>> They're relatively recent additions, but numpy.polynomial.polyvander2d and
>> numpy.polynomial.polyval2d should also do what you want, unless I'm
>> misunderstanding the problem.
>>
>>
>> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial.polyvander2d
>>
>> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial.polyval2d
>>
>> You can also do things like (you could generalize this to N-dimensions, as
>> well):
>>
>> def polyfit2d(x, y, z, order=3):
>>     ncols = (order + 1)**2
>>     G = np.zeros((x.size, ncols))
>>     ij = itertools.product(range(order+1), range(order+1))
>>     for k, (i,j) in enumerate(ij):
>>         G[:,k] = x**i * y**j
>>     m, _, _, _ = np.linalg.lstsq(G, z)
>>     return m
>>
>> def polyval2d(x, y, m):
>>     order = int(np.sqrt(len(m))) - 1
>>     ij = itertools.product(range(order+1), range(order+1))
>>     z = np.zeros_like(x)
>>     for a, (i,j) in zip(m, ij):
>>         z += a * x**i * y**j
>>     return z
>>
>>
>
> I think the "bilinear" is a mistake, as bilinear usually means terms of
> degree two. AFAICT, this question is just about multivariate linear fits
> only

like statsmodels OLS

Josef


>
> <snip>
>
> Chuck
>>
>>
>>
>
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Multivariate linear (bilinear) fit

Denis Akhiyarov

I would also recommend looking at robust regression (RLM), before you settle on OLS. Especially if the data is noisy or has outliers.  SciPy has a new regression method with bounds on the coefficients, if you get unrealistic factors from the OLS regression results, but it requires some "tuning".

Whatever you pick, looking at results in parity plots and error histograms may help as well.

http://scikit-learn.org/stable/auto_examples/linear_model/plot_ransac.html

https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.optimize.lsq_linear.html


On Wed, Apr 19, 2017, 7:47 PM <[hidden email]> wrote:
On Wed, Apr 19, 2017 at 8:12 PM, Charles R Harris
<[hidden email]> wrote:
>
>
> On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington <[hidden email]> wrote:
>>
>> They're relatively recent additions, but numpy.polynomial.polyvander2d and
>> numpy.polynomial.polyval2d should also do what you want, unless I'm
>> misunderstanding the problem.
>>
>>
>> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial.polyvander2d
>>
>> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial.polyval2d
>>
>> You can also do things like (you could generalize this to N-dimensions, as
>> well):
>>
>> def polyfit2d(x, y, z, order=3):
>>     ncols = (order + 1)**2
>>     G = np.zeros((x.size, ncols))
>>     ij = itertools.product(range(order+1), range(order+1))
>>     for k, (i,j) in enumerate(ij):
>>         G[:,k] = x**i * y**j
>>     m, _, _, _ = np.linalg.lstsq(G, z)
>>     return m
>>
>> def polyval2d(x, y, m):
>>     order = int(np.sqrt(len(m))) - 1
>>     ij = itertools.product(range(order+1), range(order+1))
>>     z = np.zeros_like(x)
>>     for a, (i,j) in zip(m, ij):
>>         z += a * x**i * y**j
>>     return z
>>
>>
>
> I think the "bilinear" is a mistake, as bilinear usually means terms of
> degree two. AFAICT, this question is just about multivariate linear fits
> only

like statsmodels OLS

Josef


>
> <snip>
>
> Chuck
>>
>>
>>
>
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user