Re: SciPy-User Digest, Vol 109, Issue 58

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: SciPy-User Digest, Vol 109, Issue 58

The Helmbolds
As a Windows-only user, and interested mainly in quickly getting my application running and generating results, I want something that installs everything I need and lets me get on with it. And I don't want to learn or use Linux along the way.

I'll be seriously discouraged if to start getting results after installing SciPy orPyLab, I next have to install A. But to install A, I first have to install B. Then I next have to install C, and then D, etc.

I also get seriously irritated when after all that I discover that to use SciPy or PyLab functions FFF and GGG, I have to make sure I've first imported modules MM1 and MM2 -- and many times scratch around until I find what modules contain FFF and the other function(s) I need. Why should _I_ do that? Why doesn't SciPy or PyLab know where it's functions are and just automatically import the right module(s)? Seems like all it would take is to add a line to each function importing the appropriate module(s).

Sent from Robaula's iPad
[hidden email]
2645 E Southern, A241
Tempe, AZ 85282
VOX: 480-831-3611
CELL: 602-568-6948 (but not often turned on)

On Sep 24, 2012, at 5:11 AM, [hidden email] wrote:

> Send SciPy-User mailing list submissions to
>    [hidden email]
>
> To subscribe or unsubscribe via the World Wide Web, visit
>    http://mail.scipy.org/mailman/listinfo/scipy-user
> or, via email, send a message with subject or body 'help' to
>    [hidden email]
>
> You can reach the person managing the list at
>    [hidden email]
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of SciPy-User digest..."
>
>
> Today's Topics:
>
>   1. Re: Pylab - standard packages (Almar Klein)
>   2. Re: Pylab - standard packages (Francesc Alted)
>   3. Re: Pylab - standard packages (Thomas Kluyver)
>   4. Re: Pylab - standard packages (Almar Klein)
>   5. Re: hyp1f1(0.5, 1.5, -2000) fails (Moore, Eric (NIH/NIDDK) [F])
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 24 Sep 2012 10:17:41 +0200
> From: Almar Klein <[hidden email]>
> Subject: Re: [SciPy-User] Pylab - standard packages
> To: SciPy Users List <[hidden email]>
> Message-ID:
>    <CAB_T6=mwrt0ac1SmXYXmm5Q1oku=[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
>>
>> We discussed it recently with Carlos Cordoba (a Spyder dev), but I'm
>> not sure what the issue is, nor whether IPython, Spyder or both need
>> to fix things. I've just opened an (IPython) issue to work this out:
>> https://github.com/ipython/ipython/issues/2425
>
>
> I realize my previous comment on this was rather vague. What I was trying
> to say is that I do not think specifying the latest IPython version in
> pylab will cause serious problems for Python(x,y). It should easily be able
> to include the IPython executable in the way that other people use it, i.e.
> independent from its IDE (Spyder).
>
> The integration of the newer IPython (or a notebook interface) in Spyder is
> much more difficult. It would be great if in time this would be available
> too, but I don't think it's a prerequisite for Python(x,y) to be
> pylab compliant.
>
> But as you pointed out, it would be nice to hear what Pierre et al think
> about this.
>
>
> Almar: thanks for the mention of imageio. By the sounds of it, that
>> makes two votes (yours and Ralf's) in favour of FreeImage over PIL?
>
>
> Well, FreeImage is only a C++ library. sk-image uses it as one of its ways
> to load images. So my take on this would be to let sk-image do the image
> loading for now, and include imagio in the standard if it is more mature.
> sk-image can then use imageio as a plugin.
>
> (note that there are one or two other wrappers around the imageio library,
> but Zach started fresh because their codebases weren't very nice.)
>
> It would be great if we can avoid including PIL in the standard, but I
> think that many people depend on it. So others might argue differently ...
>
>  Almar
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20120924/31ca759f/attachment-0001.html 
>
> ------------------------------
>
> Message: 2
> Date: Mon, 24 Sep 2012 11:46:10 +0200
> From: Francesc Alted <[hidden email]>
> Subject: Re: [SciPy-User] Pylab - standard packages
> To: [hidden email]
> Cc: NumFOCUS <[hidden email]>
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hey, nice to see this conversation going on.  I'm not currently an
> active developer of PyTables anymore (other brave people like Antonio
> Valentino, Anthony Scopatz and Josh Moore took over the project during
> the past year), but I created and lead its development for many years,
> and I still provide some feedback on the PyTables mailing list, so I'd
> be glad to contribute my view here too (although definitely it will be
> not impartial because, what the heck, I still consider it as my little
> boy ;)
>
> On 9/22/12 5:04 PM, Andrew Collette wrote:
>> On Sat, Sep 22, 2012 at 3:56 AM, Thomas Kluyver <[hidden email]> wrote:
>>
>>> Andrew: Thanks for the info about h5py. As I don't use HDF5 myself,
>>> can someone describe, as impartially as possible, the differences
>>> between PyTables and h5py: how do the APIs differ, any speed
>>> difference, how well known are they, what do they depend on, and what
>>> depends on them (e.g. I think pandas can use PyTables?). If it's
>>> sensible to include both, we can do so, but I'd like to get a feel for
>>> what they each are.
>
> PyTables is a high-level API for the HDF5 library, that includes
> advanced features that are not present in HDF5 itself, like indexing
> (via OPSI), out-of-core operations (via numexpr) and very fast
> compression (via Blosc).  PyTables, contrarily to h5py, does not try to
> expose the complete HDF5 API, and it is more centered around providing
> an easy-to-use and performant interface to the HDF5 library.
>
>> I'm certainly not unbiased, but while we're waiting for others to
>> rejoin the discussion I can give my perspective on this question.  I
>> never saw h5py and PyTables as direct competitors; they have different
>> design goals.  To me the basic difference is that PyTables is both a
>> way to talk to HDF5 and a really great database-like interface with
>> things like indexing, searching, etc. (both NumExpr and Blosc came out
>> of work on PyTables, I believe).  In contrast, h5py arose by asking
>> "how can we map the basic HDF5 abstractions to Python in a direct but
>> still Pythonic way".
>
> Yeah, I agree that the h5py and PyTables cannot be seen as direct
> competitors (although many people, including myself at times :), see
> them as such).  As I said before, performance is one of the aspects that
> is *extremely* important for PyTables, and you are right in that both
> OPSI and Blosc were developments done for the sake of PyTables.
>
> Numexpr case is somewhat different, since it is was originally developed
> outside of the project (by David M. Cooke), and adopted and largely
> enhanced for allowing fast queries and out of core computations in
> PyTables.  Of course, all these enhancements where contributed back to
> the original numexpr project and it continues to be an stand-alone
> library that is useful in many scenarios different than PyTables, in a
> typical case of fertile project cross-polinization.
>
>>
>> The API for h5py has both a high-level and low-level component; like
>> PyTables, the high-level component is oriented around files, datasets
>> and groups, allows iteration over elements in the file, etc. The
>> emphasis in h5py is to use existing objects and abstractions from
>> NumPy; for example, datasets have .dtype and .shape attributes and can
>> be sliced like NumPy arrays.  Groups are treated like dictionaries,
>> are iterable, have .keys() and .iteritems() and friends, etc.
>
> So for PyTables the approach in this regard is very close to h5py, with
> the exception that the group hierarchy is primarily (but not only)
> accessed via a natural naming approach, i.e. something like:
>
> file.root.group1.group2.dataset
>
> instead of the h5py approach:
>
> file['group1']['group2']['dataset']
>
> I find the former extremely more useful for structure discovering (by
> hitting the TAB key in REPL interactive environments), but this is
> probably a matter of tastes.
>
>>
>> The "main" high level interface in h5py also rests on a huge low-level
>> interface written in Cython
>> (http://h5py.alfven.org/docs/low/index.html), which exposes the
>> majority of the HDF5 C API in a Pythonic, object-oriented way.  The
>> goal here is anything you can do with HDF5 in C, you can do in Python.
>
> Yeah, as it has already been said, here it lies one of the big
> differences between both projects: PyTables does not come with a
> low-level interface to HDF5.  This, however, has been a deliberate
> design goal, as the HDF5 C API which can be rather complex and
> cumbersome for the large majority of Python users, and it was estimated
> that the large majority of the people was not interested in delving with
> HDF5 intricacies (those PyTables users interested in accessing to such
> HDF5 capabilities can always take the Cython sources and build new
> features on top of it, which I find a more sensible approach, specially
> if performance is interesting for the user).
>
>>
>> It has no dependencies beyond NumPy and Python itself; I will let
>> others chime in for specific projects which depend on h5py.  As a
>> rough proxy for popularity, h5py has roughly 30k downloads over the
>> life of the project (10k in the past year).
>
> I cannot tell how many downloads PyTables has had over its almost 10
> years of existence (the first public release was made back in October
> 2002), but probably a lot.  Sourceforge reports that it received more
> than 50K downloads for the 2.3 series (released one year ago) and more
> than 6.5K downloads for the recent 2.4.0 version released a couple of
> months ago.   However that's is a bit tricky because PyTables is shipped
> in most of Linux distributions, and Windows binaries are not available
> in SF anymore, but through independent Windows distributions like
> Gohlke's, EPD, Python(x,y) or Anaconda, so likely the actual number
> would be much more than that (but the same should apply to h5py).
>
>>
>> I have never benchmarked PyTables against h5py, but I strongly suspect
>> PyTables is faster.
>
> Yes, without knowing about anybody having done an exhaustive comparison
> in most of the scenarios, my own benchmarks confirm that PyTables is
> generally faster than h5py.  It is true that both projects uses HDF5 as
> the basic I/O library, but when combining things like OPSI, Blosc and
> numexpr, this can make a huge difference.  For example, in some
> benchmarks that I did some months ago, the difference in performance was
> in the range from 10 thousand to more than 100 thousand times, specially
> when browsing and querying medium-size on-disk tables (100 thousand rows
> long):
>
> http://www.digipedia.pl/usenet/thread/16009/26243/#post26257
>
> Also, Ga?l Varoquaux blogged on some real-life benchmarks about how the
> ultra-fast Blosc (and LZO) compressors integrated in PyTables can
> accelerate the I/O:
>
> http://gael-varoquaux.info/blog/?p=159
>
> Anyway, I think the home page about PyTables does a good job expressing
> how important performance is for the project:
>
> http://www.pytables.org/moin
>
>>   Most of the development effort that has recently
>> gone into h5py has been focused in other areas like API coverage,
>> Python 3 support, Unicode, and thread safety; we've never done careful
>> performance testing.
>
> Yep, probably here h5py is more advanced than PyTables, specially
> because the latter does not provide full Python 3 support yet. However,
> Antonio made great strides on this area, and most of the pieces are
> already there, being the most outstanding missing part having Python 3
> support for numexpr.  In fact, Antonio already kindly provided patches
> for this, but I still need to check them out and release a new version
> of numexpr.  I think I should stop procrastinating and give this a go
> sooner than later (Antonio, if you are reading this, be ready for some
> questions about your patches soon :).
>
> --
> Francesc Alted
>
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 24 Sep 2012 11:30:17 +0100
> From: Thomas Kluyver <[hidden email]>
> Subject: Re: [SciPy-User] Pylab - standard packages
> To: SciPy Users List <[hidden email]>
> Message-ID:
>    <[hidden email]>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Thanks everyone for your comments.
>
> Re IPython: Carlos confirms [1] that integration with Spyder is
> improving considerably. Python(x,y) will probably throw the switch on
> a newer version of IPython once Spyder 2.2 is released, which sounds
> like it will be quite soon.
>
> FreeImage: So does sk-image interface directly with the C++ library,
> or does it need some existing wrapper like FreeImagePy? From the
> sounds of it, we should include FreeImage and any wrapper sk-image
> needs, with a view to including imageio in the future. Or perhaps we
> should require FreeImage or PIL, and let distributions pick?
>
> NumFOCUS: Thanks Travis, I think the support of NumFOCUS will be
> invaluable for taking this forwards. For this discussion about the
> standard, though, I think we're benefitting from a much wider audience
> on scipy-central.
>
> HDF5: Thanks for that description, Francesc. I'm leaning towards
> specifying both PyTables and h5py. EPD, Anaconda, Python(x,y) and
> WinPython already include both. I'm not clear on whether Sage & QSnake
> do - I see no mention of HDF5 on Sage's package list. For the first
> spec version, I think we should leave the inclusion of packages to
> handle HDF4 and netCDF4 up to distributions, as it seems they're more
> specialist.
>
> [1] https://github.com/ipython/ipython/issues/2425
>
> Best wishes,
> Thomas
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 24 Sep 2012 13:36:48 +0200
> From: Almar Klein <[hidden email]>
> Subject: Re: [SciPy-User] Pylab - standard packages
> To: SciPy Users List <[hidden email]>
> Message-ID:
>    <CAB_T6=nbtedsWy1LTTD5UnacenLp7PEuXUe32ysAXg9058=[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
>>
>> FreeImage: So does sk-image interface directly with the C++ library,
>> or does it need some existing wrapper like FreeImagePy? From the
>> sounds of it, we should include FreeImage and any wrapper sk-image
>> needs, with a view to including imageio in the future. Or perhaps we
>> should require FreeImage or PIL, and let distributions pick?
>>
>
> The FreeImage plugin of sk-image interfaces directly with the library; the
> plugin and imageio are both originated from the same code (but probably
> diverged a bit by now).
>
>  Almar
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20120924/58e4f495/attachment-0001.html 
>
> ------------------------------
>
> Message: 5
> Date: Mon, 24 Sep 2012 08:15:25 -0400
> From: "Moore, Eric (NIH/NIDDK) [F]" <[hidden email]>
> Subject: Re: [SciPy-User] hyp1f1(0.5, 1.5, -2000) fails
> To: 'SciPy Users List' <[hidden email]>
> Message-ID:
>    <[hidden email]>
> Content-Type: text/plain; charset="us-ascii"
>
>> -----Original Message-----
>> From: Ondrej Certik [mailto:[hidden email]]
>> Sent: Sunday, September 23, 2012 7:32 PM
>> To: SciPy Users List
>> Subject: [SciPy-User] hyp1f1(0.5, 1.5, -2000) fails
>>
>> Hi,
>>
>> I noticed, that hyp1f1(0.5, 1.5, -2000) fails:
>>
>> In [5]: scipy.special.hyp1f1(0.5, 1.5, 0)
>> Out[5]: 1.0
>>
>> In [6]: scipy.special.hyp1f1(0.5, 1.5, -20)
>> Out[6]: 0.19816636482997368
>>
>> In [7]: scipy.special.hyp1f1(0.5, 1.5, -200)
>> Out[7]: 0.062665706865775023
>>
>> In [8]: scipy.special.hyp1f1(0.5, 1.5, -2000)
>> Warning: invalid value encountered in hyp1f1
>> Out[8]: nan
>>
>>
>> The values [5], [6] and [7] are correct. The value [8] should be:
>>
>> 0.019816636488030055...
>>
>> Ondrej
>> _______________________________________________
>> SciPy-User mailing list
>> [hidden email]
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>
> Its blowing up around -709:
>
> In [60]: s.hyp1f1(0.5, 1.5, -709.7827128933)
> Out[60]: 0.03326459435722777
>
> In [61]: s.hyp1f1(0.5, 1.5, -709.7827128934)
> Out[61]: inf
>
> Eric
>
>
> ------------------------------
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
> End of SciPy-User Digest, Vol 109, Issue 58
> *******************************************
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy-User Digest, Vol 109, Issue 58

Robert Kern-2
On Mon, Sep 24, 2012 at 9:57 AM, Robaula <[hidden email]> wrote:

> I also get seriously irritated when after all that I discover that to use SciPy or PyLab functions FFF and GGG, I have to make sure I've first imported modules MM1 and MM2 -- and many times scratch around until I find what modules contain FFF and the other function(s) I need. Why should _I_ do that? Why doesn't SciPy or PyLab know where it's functions are and just automatically import the right module(s)? Seems like all it would take is to add a line to each function importing the appropriate module(s).

Can you provide specific examples of this? I know of nothing that
works like this.

--
Robert Kern
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy-User Digest, Vol 109, Issue 58

Thomas Kluyver-2
In reply to this post by The Helmbolds
On 24 September 2012 15:57, Robaula <[hidden email]> wrote:
> I'll be seriously discouraged if to start getting results after installing SciPy orPyLab, I next have to install A. But to install A, I first have to install B. Then I next have to install C, and then D, etc.

Well, that is the sort of thing we're aiming to improve. But equally,
we can't know everything you're going to want to use, so depending on
what you want to do, you might still have to install one or more extra
packages.

> And I don't want to learn or use Linux along the way.

Ironically, this is precisely what Linux distributions do very well:
dependency management. So when I ask it to install A, it works out
that B, C and D are needed, downloads and installs them in the right
order. Package management systems like APT are miles ahead of the
Python packaging stuff.

Thomas
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy-User Digest, Vol 109, Issue 58

josef.pktd
In reply to this post by The Helmbolds
On Mon, Sep 24, 2012 at 10:57 AM, Robaula <[hidden email]> wrote:
> As a Windows-only user, and interested mainly in quickly getting my application running and generating results, I want something that installs everything I need and lets me get on with it. And I don't want to learn or use Linux along the way.
>
> I'll be seriously discouraged if to start getting results after installing SciPy orPyLab, I next have to install A. But to install A, I first have to install B. Then I next have to install C, and then D, etc.
>
> I also get seriously irritated when after all that I discover that to use SciPy or PyLab functions FFF and GGG, I have to make sure I've first imported modules MM1 and MM2 -- and many times scratch around until I find what modules contain FFF and the other function(s) I need. Why should _I_ do that? Why doesn't SciPy or PyLab know where it's functions are and just automatically import the right module(s)? Seems like all it would take is to add a line to each function importing the appropriate module(s).

Because the program/package cannot or should not guess which function you want.
scipy.linalg or numpy.linalg
numpy.any or python's any
and so on.


when you load some R packages, then you get warnings like, this and
this function has been overwritten by the package that has been
loaded.
In Stata there is a chapter describing the search path and the priority.
Similar in matlab, I always had to guess where the function is coming
from and if the function I wanted hasn't been replaced by something
else. And I was careful, adding a prefix to my function names, so I
don't make mistakes.

names spaces are one honking ...
>>> import this

The windows help file makes searching and finding the location of a
function very easy.

Josef

>
> Sent from Robaula's iPad
> [hidden email]
> 2645 E Southern, A241
> Tempe, AZ 85282
> VOX: 480-831-3611
> CELL: 602-568-6948 (but not often turned on)
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user