# [SciPy-User] Multivariate linear (bilinear) fit

8 messages
Open this post in threaded view
|
Report Content as Inappropriate

## [SciPy-User] Multivariate linear (bilinear) fit

 Hello, before I write my own, does SciPy come with a function for multivariate (often also called bilinear, in the case of two variables) linear fits? I would like to fit a function if the form:   y = c + p1 * x1 + p2 * x2 + ... + pn * xn which is a simple least squares problem, but I would like to avoid to write the code :-) Thanks. Cheers, Daniele _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user
Open this post in threaded view
|
Report Content as Inappropriate

## Re: Multivariate linear (bilinear) fit

 Hi Daniele,Do you know lmfit (https://lmfit.github.io/lmfit-py/)? It's really simple to use and very well documented. Cheers,PawelPaweł 2017-04-18 19:17 GMT+02:00 Daniele Nicolodi :Hello, before I write my own, does SciPy come with a function for multivariate (often also called bilinear, in the case of two variables) linear fits? I would like to fit a function if the form:   y = c + p1 * x1 + p2 * x2 + ... + pn * xn which is a simple least squares problem, but I would like to avoid to write the code :-) Thanks. Cheers, Daniele _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user
Open this post in threaded view
|
Report Content as Inappropriate

## Re: Multivariate linear (bilinear) fit

 ---------------------------------------------------------------------- Message: 1 Date: Tue, 18 Apr 2017 11:17:26 -0600 From: Daniele Nicolodi <[hidden email]> To: [hidden email] Subject: [SciPy-User] Multivariate linear (bilinear) fit Message-ID: <[hidden email]> Content-Type: text/plain; charset=utf-8 Hello, before I write my own, does SciPy come with a function for multivariate (often also called bilinear, in the case of two variables) linear fits? I would like to fit a function if the form: y = c + p1 * x1 + p2 * x2 + ... + pn * xn which is a simple least squares problem, but I would like to avoid to write the code :-) Thanks. Cheers, Daniele ===================== Take a look at the optimization tools from SciPy     http://www.scipy-lectures.org/advanced/mathematical_optimization/index.htmlSergio Enhance your #MachineLearning and #BigData skills via #Python #SciPy: 1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition_______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user
Open this post in threaded view
|
Report Content as Inappropriate

## Re: Multivariate linear (bilinear) fit

 In reply to this post by Daniele Nicolodi On Tue, Apr 18, 2017 at 10:17 AM, Daniele Nicolodi <[hidden email]> wrote: > Hello, > > before I write my own, does SciPy come with a function for multivariate > (often also called bilinear, in the case of two variables) linear fits? > > I would like to fit a function if the form: > >   y = c + p1 * x1 + p2 * x2 + ... + pn * xn > > which is a simple least squares problem, but I would like to avoid to > write the code :-) np.linalg.lstsq and scipy.linalg.lstsq handle both univariate and multivariate problems. They're pretty bare-bones, though, and they also compute the sum of squared residuals which can make them slow if you don't need it and are solving lots of problems in a tight loop. -n -- Nathaniel J. Smith -- https://vorpus.org_______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user
Open this post in threaded view
|
Report Content as Inappropriate

## Re: Multivariate linear (bilinear) fit

 They're relatively recent additions, but numpy.polynomial.polyvander2d and numpy.polynomial.polyval2d should also do what you want, unless I'm misunderstanding the problem.You can also do things like (you could generalize this to N-dimensions, as well):``````def polyfit2d(x, y, z, order=3): ncols = (order + 1)**2 G = np.zeros((x.size, ncols)) ij = itertools.product(range(order+1), range(order+1)) for k, (i,j) in enumerate(ij): G[:,k] = x**i * y**j m, _, _, _ = np.linalg.lstsq(G, z) return m def polyval2d(x, y, m): order = int(np.sqrt(len(m))) - 1 ij = itertools.product(range(order+1), range(order+1)) z = np.zeros_like(x) for a, (i,j) in zip(m, ij): z += a * x**i * y**j return z``````On Wed, Apr 19, 2017 at 4:17 PM, Nathaniel Smith wrote:On Tue, Apr 18, 2017 at 10:17 AM, Daniele Nicolodi <[hidden email]> wrote: > Hello, > > before I write my own, does SciPy come with a function for multivariate > (often also called bilinear, in the case of two variables) linear fits? > > I would like to fit a function if the form: > >   y = c + p1 * x1 + p2 * x2 + ... + pn * xn > > which is a simple least squares problem, but I would like to avoid to > write the code :-) np.linalg.lstsq and scipy.linalg.lstsq handle both univariate and multivariate problems. They're pretty bare-bones, though, and they also compute the sum of squared residuals which can make them slow if you don't need it and are solving lots of problems in a tight loop. -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user
Open this post in threaded view
|
Report Content as Inappropriate

## Re: Multivariate linear (bilinear) fit

 On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington wrote:They're relatively recent additions, but numpy.polynomial.polyvander2d and numpy.polynomial.polyval2d should also do what you want, unless I'm misunderstanding the problem.You can also do things like (you could generalize this to N-dimensions, as well):``````def polyfit2d(x, y, z, order=3): ncols = (order + 1)**2 G = np.zeros((x.size, ncols)) ij = itertools.product(range(order+1), range(order+1)) for k, (i,j) in enumerate(ij): G[:,k] = x**i * y**j m, _, _, _ = np.linalg.lstsq(G, z) return m def polyval2d(x, y, m): order = int(np.sqrt(len(m))) - 1 ij = itertools.product(range(order+1), range(order+1)) z = np.zeros_like(x) for a, (i,j) in zip(m, ij): z += a * x**i * y**j return z``````I think the "bilinear" is a mistake, as bilinear usually means terms of degree two. AFAICT, this question is just about multivariate linear fits onlyChuck _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user
Open this post in threaded view
|
Report Content as Inappropriate

## Re: Multivariate linear (bilinear) fit

 On Wed, Apr 19, 2017 at 8:12 PM, Charles R Harris <[hidden email]> wrote: > > > On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington <[hidden email]> wrote: >> >> They're relatively recent additions, but numpy.polynomial.polyvander2d and >> numpy.polynomial.polyval2d should also do what you want, unless I'm >> misunderstanding the problem. >> >> >> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial.polyvander2d>> >> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial.polyval2d>> >> You can also do things like (you could generalize this to N-dimensions, as >> well): >> >> def polyfit2d(x, y, z, order=3): >>     ncols = (order + 1)**2 >>     G = np.zeros((x.size, ncols)) >>     ij = itertools.product(range(order+1), range(order+1)) >>     for k, (i,j) in enumerate(ij): >>         G[:,k] = x**i * y**j >>     m, _, _, _ = np.linalg.lstsq(G, z) >>     return m >> >> def polyval2d(x, y, m): >>     order = int(np.sqrt(len(m))) - 1 >>     ij = itertools.product(range(order+1), range(order+1)) >>     z = np.zeros_like(x) >>     for a, (i,j) in zip(m, ij): >>         z += a * x**i * y**j >>     return z >> >> > > I think the "bilinear" is a mistake, as bilinear usually means terms of > degree two. AFAICT, this question is just about multivariate linear fits > only like statsmodels OLS Josef > > > > Chuck >> >> >> > > > _______________________________________________ > SciPy-User mailing list > [hidden email] > https://mail.python.org/mailman/listinfo/scipy-user> _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user
Open this post in threaded view
|
Report Content as Inappropriate

## Re: Multivariate linear (bilinear) fit

 I would also recommend looking at robust regression (RLM), before you settle on OLS. Especially if the data is noisy or has outliers.  SciPy has a new regression method with bounds on the coefficients, if you get unrealistic factors from the OLS regression results, but it requires some "tuning". Whatever you pick, looking at results in parity plots and error histograms may help as well. http://scikit-learn.org/stable/auto_examples/linear_model/plot_ransac.html https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.optimize.lsq_linear.html On Wed, Apr 19, 2017, 7:47 PM <[hidden email]> wrote:On Wed, Apr 19, 2017 at 8:12 PM, Charles R Harris <[hidden email]> wrote: > > > On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington <[hidden email]> wrote: >> >> They're relatively recent additions, but numpy.polynomial.polyvander2d and >> numpy.polynomial.polyval2d should also do what you want, unless I'm >> misunderstanding the problem. >> >> >> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial.polyvander2d >> >> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial.polyval2d >> >> You can also do things like (you could generalize this to N-dimensions, as >> well): >> >> def polyfit2d(x, y, z, order=3): >>     ncols = (order + 1)**2 >>     G = np.zeros((x.size, ncols)) >>     ij = itertools.product(range(order+1), range(order+1)) >>     for k, (i,j) in enumerate(ij): >>         G[:,k] = x**i * y**j >>     m, _, _, _ = np.linalg.lstsq(G, z) >>     return m >> >> def polyval2d(x, y, m): >>     order = int(np.sqrt(len(m))) - 1 >>     ij = itertools.product(range(order+1), range(order+1)) >>     z = np.zeros_like(x) >>     for a, (i,j) in zip(m, ij): >>         z += a * x**i * y**j >>     return z >> >> > > I think the "bilinear" is a mistake, as bilinear usually means terms of > degree two. AFAICT, this question is just about multivariate linear fits > only like statsmodels OLS Josef > > > > Chuck >> >> >> > > > _______________________________________________ > SciPy-User mailing list > [hidden email] > https://mail.python.org/mailman/listinfo/scipy-user > _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user _______________________________________________ SciPy-User mailing list [hidden email] https://mail.python.org/mailman/listinfo/scipy-user