Re: SciPy-User Digest, Vol 109, Issue 66

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: SciPy-User Digest, Vol 109, Issue 66

The Helmbolds
Denis Laxalde writes:

...snip...

> So what's wrong? Am I missing something?

I think so. I think you're missing line 165 of the fmin_cobyla routine.

>  This is what the optimize tutorial [1] is for, I guess.
> 1: http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html

Inadequate and IMHO, useless. Also, doesn't mention fmin_cobyla, cobyla, or COBYLA.

Bob H
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Flaws in fmin_cobyla

Moore, Eric (NIH/NIDDK) [F]
> -----Original Message-----
> From: Robaula [mailto:[hidden email]]
> Sent: Thursday, September 27, 2012 3:48 PM
> To: [hidden email]
> Subject: Re: [SciPy-User] SciPy-User Digest, Vol 109, Issue 66
>
> Denis Laxalde writes:
>
> ...snip...
>
> > So what's wrong? Am I missing something?
>
> I think so. I think you're missing line 165 of the fmin_cobyla routine.
>
> >  This is what the optimize tutorial [1] is for, I guess.
> > 1: http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html
>
> Inadequate and IMHO, useless. Also, doesn't mention fmin_cobyla,
> cobyla, or COBYLA.
>
> Bob H
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user

The various data produced by fmin_cobyla routine are printed using a direct call to PRINT from within the fortran routine. This is less than optimal because if you are not running in a terminal (ie, at python, or ipython) you won't see any of the output.

So try executing the example or changing the disp parameter while running from the console, and they will work as expected.

I'd say this is a big gotcha that should be noted in the docs at least.  The better choice would really be to patch cobyla2.f so that this would work even in the ipython qtconsole or wherever Bob is running his code.

If there really is something to line 165 (https://github.com/scipy/scipy/blob/master/scipy/optimize/cobyla.py#L165) it's not obvious to me.  Could you elaborate?

Eric
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Flaws in fmin_cobyla

Joe Harrington
Robaula <[hidden email]> wrote:

> However, I think there other, and more important issues here, namely,
> "How should the new 'minimize' interfaces be documented? How should
> existing documentation for the "old" interfaces be changed in light of
> the new 'minimize' approach? How to bring the two of them more nearly
> into alignment? Are changes to the documentation standards advisable to
> facilitate this?" Sure, "minimize's" docstring is an excellent start,
> tho I do see a couple areas where a little more explanation and/or some
> examples would be helpful. Among other things, I think it would be a lot
> better to just come out and tell folks what options are available for
> each of the various "methods", rather than telling them the code
> sequence to use to view them. Certainly the cobyla options for
> 'minimize' differ noticeably from those for the fmin_cobyla
> approach. For example, using fmin_cobyla ignores disp and iprint and
> never returns 'Results', while using 'minimize' has no iprint option,
> ignores the disp option, a nd always returns 'Results'.
>
> I'd welcome good ideas on how best to document the new 'minimize'
> interface, and how best to coordinate that with the "old style" method
> documentation. I'm not looking for ideas that are merely the
> programmer's personal reminders--the current documentation is just fine
> for that! Instead, I'm looking for documentation suitable for use by
> folks who might be described as: first year grad students just slightly
> familiar with Python who have a one week deadline to solve their problem
> and desperately need _very_ clear, unambiguous, and detailed
> instructions on exactly how to proceed. Try to remember _your_ first
> year grad experiences!!

A couple of things to remember about doc writing.

First, we developed the current docstring standard mainly with numpy in
mind.  NumPy's structure is a lot flatter than SciPy's, and we
recognized that some additional kinds of doc pages might (or might not)
be needed for it.  The Matplotlib folks made some adjustments (with our
blessing, not that they needed it) to allow the central documentation of
large sets of parameters shared by many functions (e.g., for line
thickness, color, style, etc.), something that doesn't happen in NumPy.
So, if SciPy needs something special because of its structure that isn't
covered in the standard, please bring it up on the scipy-dev mailing
list, where such discussions occur.

Rather than being a large, flat collection of loosely related and mostly
non-interacting functions, most/all of SciPy is arranged in deep modules
with lots of underlying structure and shared concepts.  It does not make
sense to document such structures only on the pages of a few functions,
nor on the page of every function.  Rather, you want to push that up to
the module level, and you can augment that with topical doc pages that
hang off the module, as long as they are referenced from the module doc
and those of any functions affected.  That way, the doc of each function
only contains what's special to that function, and refers to the module
doc for the overview and more info.

...and all THAT said, remember that this is REFERENCE documentation.  It
should talk about how the package is set up and how to get things done
in it.  It needs to give examples that distinguish cases from one
another and to assist with basic use.  However, exercises, descriptions
of what this area of math is for, philosophy, and extended examples
involving much more than the minimum complexity to make a point belong
in a tutorial dcoument, not a reference document.  The reference docs
are not the place to get a math education.  Actually, the tutorial docs
are not the best place for that, either, though they do sometimes play
that role.  Rather, a good doc writer will identify several good, FREE,
online sources, often including a Wikipedia article (but READ the
article, they're not uniformly good or correct), as well as some
WIDELY-USED textbook sections, and put those in the references section
of the docs.

--jh--
Joe Harrington
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Flaws in fmin_cobyla

Pauli Virtanen-3
In reply to this post by Moore, Eric (NIH/NIDDK) [F]
27.09.2012 23:14, Moore, Eric (NIH/NIDDK) [F] kirjoitti:
>> -----Original Message-----
>> From: Robaula [mailto:[hidden email]]
[clip]
>> I think so. I think you're missing line 165 of the fmin_cobyla routine.
[clip]

> The various data produced by fmin_cobyla routine are printed using
> a direct call to PRINT from within the fortran routine. This is less
> than optimal because if you are not running in a terminal (ie, at
> python, or ipython) you won't see any of the output.
> So try executing the example or changing the disp parameter
> while running from the console, and they will work as expected.
>
> I'd say this is a big gotcha that should be noted in the docs at least.
> The better choice would really be to patch cobyla2.f so that this would
> work even in the ipython qtconsole or wherever Bob is running his code.
>
> If there really is something to line 165
> (https://github.com/scipy/scipy/blob/master/scipy/optimize/cobyla.py#L165)
> it's not obvious to me.  Could you elaborate?

Both iprint and disp are passed correctly down to the Fortran routine.

Fixing the issue with converting output from stdout to Python's
sys.stdout can be done by passing a Python callback function down to the
fortran code, and replacing prints with it + some Fortran string
processing. There are also a couple of other cases of this, in Qhull and
some of the integrators in scipy.integrate.

Ideally, Ipython consoles et al. would redirect stdout/stderr file
handles so that also the output that comes outside from Python land
would be shown to the user. However, the correct behavior for Scipy
routines would be to print via Python's I/O.

--
Pauli Virtanen



_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Flaws in fmin_cobyla

Nathaniel Smith
On Thu, Sep 27, 2012 at 9:57 PM, Pauli Virtanen <[hidden email]> wrote:
> Ideally, Ipython consoles et al. would redirect stdout/stderr file
> handles so that also the output that comes outside from Python land
> would be shown to the user.

This is possible: redirect the underlying file descriptors to local
pipes, then spawn threads to read those pipes and call
sys.std{out,err}.write:
http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/redirecting-standard-io.html

Maybe someone should file a bug on IPython...

-n
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Flaws in fmin_cobyla

Thomas Kluyver-2
On 27 September 2012 22:58, Nathaniel Smith <[hidden email]> wrote:
> This is possible: redirect the underlying file descriptors to local
> pipes, then spawn threads to read those pipes and call
> sys.std{out,err}.write:
> http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/redirecting-standard-io.html
>
> Maybe someone should file a bug on IPython...

We already have one open: ;-)
https://github.com/ipython/ipython/issues/1230

Thanks for the link, I'll add it to that issue.

Thomas
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user