[SciPy-User] optimize.minimize - help me understand arrays as variables

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] optimize.minimize - help me understand arrays as variables

KURT PETERS
 I'm trying to use scipy.optimize.minimize.
  I've tried multiple "multivariate" methods that don't seem to actually take multivariate data and derivatives.  Can someone tell me how I can make the multivariate part of the solver actually work?
 
Here's an example:
My main function the following (typical length for N is 3):
 
  input guess is a x0=np.array([1,2,3])
  the optimization function returns:
  def calc_f3d(...):
      f3d = np.ones((np.max([3,N]),1)
      .... do some assignments to f3d[row,0] ....
      return np.linalg.norm(f3d)   # numpy.array that's 3x1
 
The jacobian returns a Nx3 matrix:
def jacob3d(...):
    df = np.ones((np.max([3,N]),3))
    ... do some assignments to df[row,col]
    return df               # note numpy.array that's 3x3
 
The optimize call is:
    OptimizeResult = optimize.minimize(
        fun=tdcalc.calc_f3d,
        x0=ract,
        jac=tdcalc.jacob3d,
        method='BFGS',
        args=(operdata,),
        tol=1.0e-8,
        options={'maxiter': 40000, 'xtol':1e-8})  <--- ops change based on whether using Newton-CG or BFGS
 
When I use BFGS, I get:
Traceback (most recent call last):
  File "./tdoa_calc.py", line 664, in <module>
    options={'maxiter': 40000, 'gtol':1e-8})
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/_minimize.py", line 348, in minimize
    return _minimize_bfgs(fun, x0, args, jac, callback, **options)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/optimize.py", line 779, in _minimize_bfgs
    old_fval, old_old_fval)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/linesearch.py", line 95, in line_search_wolfe1
    c1=c1, c2=c2, amax=amax, amin=amin, xtol=xtol)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/linesearch.py", line 147, in scalar_search_wolfe1
    alpha1 = min(1.0, 1.01*2*(phi0 - old_phi0)/derphi0)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
 
When I use Newton-CG, I get:
Traceback (most recent call last):
  File "./tdoa_calc.py", line 655, in <module>
    options={'maxiter': 40000, 'xtol':1e-8})
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/_minimize.py", line 351, in minimize
    **options)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/optimize.py", line 1320, in _minimize_newtoncg
    eta = numpy.min([0.5, numpy.sqrt(maggrad)])
  File "/usr/lib64/python2.7/site-packages/numpy/core/fromnumeric.py", line 1982, in amin
    out=out, keepdims=keepdims)
  File "/usr/lib64/python2.7/site-packages/numpy/core/_methods.py", line 14, in _amin
    out=out, keepdims=keepdims)
ValueError: setting an array element with a sequence.

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: optimize.minimize - help me understand arrays as variables

Andrew Nelson
`calc_f3d` needs to return a single number, the overall 'cost'.

On 12 January 2015 at 11:55, KURT PETERS <[hidden email]> wrote:
 I'm trying to use scipy.optimize.minimize.
  I've tried multiple "multivariate" methods that don't seem to actually take multivariate data and derivatives.  Can someone tell me how I can make the multivariate part of the solver actually work?
 
Here's an example:
My main function the following (typical length for N is 3):
 
  input guess is a x0=np.array([1,2,3])
  the optimization function returns:
  def calc_f3d(...):
      f3d = np.ones((np.max([3,N]),1)
      .... do some assignments to f3d[row,0] ....
      return np.linalg.norm(f3d)   # numpy.array that's 3x1
 
The jacobian returns a Nx3 matrix:
def jacob3d(...):
    df = np.ones((np.max([3,N]),3))
    ... do some assignments to df[row,col]
    return df               # note numpy.array that's 3x3
 
The optimize call is:
    OptimizeResult = optimize.minimize(
        fun=tdcalc.calc_f3d,
        x0=ract,
        jac=tdcalc.jacob3d,
        method='BFGS',
        args=(operdata,),
        tol=1.0e-8,
        options={'maxiter': 40000, 'xtol':1e-8})  <--- ops change based on whether using Newton-CG or BFGS
 
When I use BFGS, I get:
Traceback (most recent call last):
  File "./tdoa_calc.py", line 664, in <module>
    options={'maxiter': 40000, 'gtol':1e-8})
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/_minimize.py", line 348, in minimize
    return _minimize_bfgs(fun, x0, args, jac, callback, **options)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/optimize.py", line 779, in _minimize_bfgs
    old_fval, old_old_fval)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/linesearch.py", line 95, in line_search_wolfe1
    c1=c1, c2=c2, amax=amax, amin=amin, xtol=xtol)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/linesearch.py", line 147, in scalar_search_wolfe1
    alpha1 = min(1.0, 1.01*2*(phi0 - old_phi0)/derphi0)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
 
When I use Newton-CG, I get:
Traceback (most recent call last):
  File "./tdoa_calc.py", line 655, in <module>
    options={'maxiter': 40000, 'xtol':1e-8})
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/_minimize.py", line 351, in minimize
    **options)
  File "/usr/lib64/python2.7/site-packages/scipy/optimize/optimize.py", line 1320, in _minimize_newtoncg
    eta = numpy.min([0.5, numpy.sqrt(maggrad)])
  File "/usr/lib64/python2.7/site-packages/numpy/core/fromnumeric.py", line 1982, in amin
    out=out, keepdims=keepdims)
  File "/usr/lib64/python2.7/site-packages/numpy/core/_methods.py", line 14, in _amin
    out=out, keepdims=keepdims)
ValueError: setting an array element with a sequence.

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user




--
_____________________________________
Dr. Andrew Nelson


_____________________________________

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: optimize.minimize - help me understand arrays as variables

KURT PETERS
See below

> Date: Thu, 15 Jan 2015 11:01:56 +1100
> From: Geordie McBain <[hidden email]>
> Subject: Re: [SciPy-User] optimize.minimize - help me understand
> arrays as variables (Andrew Nelson)
> To: SciPy Users List <[hidden email]>
> Message-ID:
> <CAB0gBH38Zx4Sp=zxxPVe_RjTqJ+ifHTQikW3WwLRYt+4o1Y1=[hidden email]>
> Content-Type: text/plain; charset=UTF-8
>
> 2015-01-13 10:26 GMT+11:00 KURT PETERS <[hidden email]>:
> > > Date: Sun, 11 Jan 2015 17:55:32 -0700
> >> From: KURT PETERS <[hidden email]>
> >> Subject: [SciPy-User] optimize.minimize - help me understand arrays as
> >> variables
> >> To: "[hidden email]" <[hidden email]>
> >> Message-ID: <[hidden email]>
> >> Content-Type: text/plain; charset="iso-8859-1"
> >>
> >> I'm trying to use scipy.optimize.minimize.
> >> I've tried multiple "multivariate" methods that don't seem to actually
> >> take multivariate data and derivatives. Can someone tell me how I can make
> >> the multivariate part of the solver actually work?
> >>
> >> Here's an example:
> >> My main function the following (typical length for N is 3):
> >>
> >> input guess is a x0=np.array([1,2,3])
> >> the optimization function returns:
> >> def calc_f3d(...):
> >> f3d = np.ones((np.max([3,N]),1)
> >> .... do some assignments to f3d[row,0] ....
> >> return np.linalg.norm(f3d) # numpy.array that's 3x1
> >>
> >> The jacobian returns a Nx3 matrix:
> >> def jacob3d(...):
> >> df = np.ones((np.max([3,N]),3))
> >> ... do some assignments to df[row,col]
> >> return df # note numpy.array that's 3x3
>
> Hello. I think that this might be the problem here: the jac should
> have the same shape as x, i.e. (N,), not (N, 3); the components of the
> jac are the partial derivatives of fun with respect to the
> corresponding components of x. Think of it as the gradient of the
> objective.
>
> Here's a simple shape=(2,) example, taken from D. M. Greig's
> Optimisation (1980, London: Longman). The exact minimum is 0 at [1,
> 1].
>
> def f(x): # Greig (1980, p. 48)
> return (x[1] - x[0]**2)**2 + (1 - x[0])**2
>
> def g(x): # ibid
> return np.array([-4*x[0]*(x[1] - x[0]**2) - 2 + 2*x[0],
> 2 * (x[1] - x[0]**2)])
>
> x = np.zeros(2)
> print('Without Jacobian: ', minimize(f, x))
> print('\nWith:', minimize(f, x, jac=g))

Geordie,
I don't think that's the case.  Everything I've ever learned about the Jacobian is that it's the partials of each function with respect to each variable... so two equations with two unknowns, would yield a 2x2.  Here's a wiki explaining what I mean:
http://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant
 
 If what you're saying is right, then the people that developed the function don't know what a Jacobian is.  I would find that hard to believe.
 
Kurt
 
 
 

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: optimize.minimize - help me understand arrays as variables

Daπid
On 15 January 2015 at 16:00, KURT PETERS <[hidden email]> wrote:
> Geordie,
> I don't think that's the case.  Everything I've ever learned about the
> Jacobian is that it's the partials of each function with respect to each
> variable... so two equations with two unknowns, would yield a 2x2.

Ah, but for a minimisation problem you can only have one equation. You
can't take the minimum of a vector function, because you can't say one
vector is smaller than the other (or, in other words, there are many
possible definitions of smaller, and you have to choose). |R^n is not
ordered.

In your case, your function to be minimised returns
np.linalg.norm(f3d), that is, a single number, f(\vec x). Therefore,
the jacobian is a vector: [d f(\vec x)/d x_0, d f(\vec x)/d x_1, d
f(\vec x)/d x_2].

I believe your results are incorrect because how you are defining f3d:

  f3d = np.ones((np.max([3,N]),1)

There is no need for the extra dimension, just

   f3d = np.ones(max(3, N))

And do the  assignments to f3d[row]


/David.
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user