[SciPy-User] SciPy ecosystem and Python 3

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] SciPy ecosystem and Python 3

Thomas Kluyver-2
At a conversation over lunch here at the SciPy conference, a few of us mentioned that we're starting to use Python 3 in earnest for our work.

For new users, the choice of two major Python versions is confusing and offputting, and we're not going to completely get rid of that confusion until we can simply point new users to Python 3. Most of our introductions, like the SciPy stack install page, point to Python 2 because of the ecosystem, but more and more packages now support Python 3, and we're reaching the point where we could reasonably recommend Python 3 for new users.

The aim of this post is to get an overview of where the ecosystem is with:
- What packages don't yet support Python 3, or are still too unstable?
- How important are each of those: how widely relevant are they, and are substitutes available?
- What other conditions need to be met to recommend Python 3? E.g. Scientific Python distros, Linux distro packaging, documentation, etc.

Thanks,
Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

ralfgommers



On Fri, Jun 28, 2013 at 2:22 AM, Thomas Kluyver <[hidden email]> wrote:
At a conversation over lunch here at the SciPy conference, a few of us mentioned that we're starting to use Python 3 in earnest for our work.

For new users, the choice of two major Python versions is confusing and offputting, and we're not going to completely get rid of that confusion until we can simply point new users to Python 3. Most of our introductions, like the SciPy stack install page, point to Python 2 because of the ecosystem, but more and more packages now support Python 3, and we're reaching the point where we could reasonably recommend Python 3 for new users.

The aim of this post is to get an overview of where the ecosystem is with:
- What packages don't yet support Python 3, or are still too unstable?

scikit-learn
 
- How important are each of those: how widely relevant are they, and are substitutes available?
- What other conditions need to be met to recommend Python 3? E.g. Scientific Python distros, Linux distro packaging, documentation, etc.

Before recommending Python 3.x over 2.x I think it's important to not only have the very latest release or master branch of projects support 3.x, but at least 1 or 2 more versions. Reason: a lot of users (I suspect the majority) will not be able to freely upgrade to the latest version of projects.

Packaging and documentation are of course also important. No 2to3 perhaps desirable.

I think we're still one to two years away from recommending 3.x over 2.x

Ralf


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Thomas Kluyver-2
> scikit-learn

Yes, I saw that the sprint here in Austin was planning to work on Py3 support. Olivier, how did that go?

On 29 June 2013 22:47, Ralf Gommers <[hidden email]> wrote:
Before recommending Python 3.x over 2.x I think it's important to not only have the very latest release or master branch of projects support 3.x, but at least 1 or 2 more versions. Reason: a lot of users (I suspect the majority) will not be able to freely upgrade to the latest version of projects.

I certainly wouldn't count a project as having Py3 support until there's a released version. But if they can't upgrade to the latest version, chances are that they also don't have a choice between Py2 and Py3, so our recommendation doesn't matter much to them. In that case, the recommendation is targeting the sysadmin who will be deciding what to install next year.
 
Packaging and documentation are of course also important. No 2to3 perhaps desirable.

Packaging: Debian/Ubuntu already have all the core SciPy stack packages except sympy for Python 3 (and I'm working on getting sympy done). Do we know where other distros are?

Docs: I suspect there's still some way to go, but a lot of that will probably be quite mechanical print -> print(). Some of this will have to be part of the 'Python 3 D-day', because we probably don't want to make all the docs assume Python 3 while we're still recommending Python 2.

No 2to3: Desirable, but not essential, I think. It's more of a developer problem than a user problem.

Thanks,
Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Olivier Grisel-3
2013/6/29 Thomas Kluyver <[hidden email]>:
>> scikit-learn
>
> Yes, I saw that the sprint here in Austin was planning to work on Py3
> support. Olivier, how did that go?

We made some progress but it's not complete yet. I had to work on
other issues as well.

The Python 3 port of scikit-learn will probably get completed during
the Paris sprint in July though.

--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Joon Ro
In reply to this post by Thomas Kluyver-2
On Thu, Jun 27, 2013 at 7:22 PM, Thomas Kluyver <[hidden email]> wrote:
- What other conditions need to be met to recommend Python 3? E.g. Scientific Python distros, Linux distro packaging, documentation, etc.


For me having a cross-platform scientific python distro with python 3 would make a big difference - because even with Python 2, individually installing different scientific packages can be challenging. 

Also with a distro it would be much easier to check if some of the libraries one uses are available or not - I don't want keep checking this for each library that I use, it would be easier to just check a disto's available library list. Also with a distro I can selectively start using Python 3 for the projects which does not depend on missing libraries.

-Joon


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Travis Oliphant-6

Anaconda makes it easy to create an environment with Python 3 packages.   Many are already available.  

Try

conda create -n py33 python=3.3 numpy

look at anaconda packages to see all the available free binaries for python 3.3

On Jun 29, 2013 6:48 PM, "Joon Ro" <[hidden email]> wrote:
On Thu, Jun 27, 2013 at 7:22 PM, Thomas Kluyver <[hidden email]> wrote:
- What other conditions need to be met to recommend Python 3? E.g. Scientific Python distros, Linux distro packaging, documentation, etc.


For me having a cross-platform scientific python distro with python 3 would make a big difference - because even with Python 2, individually installing different scientific packages can be challenging. 

Also with a distro it would be much easier to check if some of the libraries one uses are available or not - I don't want keep checking this for each library that I use, it would be easier to just check a disto's available library list. Also with a distro I can selectively start using Python 3 for the projects which does not depend on missing libraries.

-Joon


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Joon Ro


On Sat, Jun 29, 2013 at 7:01 PM, Travis Oliphant <[hidden email]> wrote:

Anaconda makes it easy to create an environment with Python 3 packages.   Many are already available.  

Try

conda create -n py33 python=3.3 numpy

look at anaconda packages to see all the available free binaries for python 3.3


Thanks! I will give it a try.

-Joon

 
On Jun 29, 2013 6:48 PM, "Joon Ro" <[hidden email]> wrote:
On Thu, Jun 27, 2013 at 7:22 PM, Thomas Kluyver <[hidden email]> wrote:
- What other conditions need to be met to recommend Python 3? E.g. Scientific Python distros, Linux distro packaging, documentation, etc.


For me having a cross-platform scientific python distro with python 3 would make a big difference - because even with Python 2, individually installing different scientific packages can be challenging. 

Also with a distro it would be much easier to check if some of the libraries one uses are available or not - I don't want keep checking this for each library that I use, it would be easier to just check a disto's available library list. Also with a distro I can selectively start using Python 3 for the projects which does not depend on missing libraries.

-Joon


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Thomas Kluyver-2
In reply to this post by Joon Ro
On 30 June 2013 00:48, Joon Ro <[hidden email]> wrote:
For me having a cross-platform scientific python distro with python 3 would make a big difference - because even with Python 2, individually installing different scientific packages can be challenging. 

Almar Klein also has his Pyzo distro based on Python 3. The list of packages can be seen here, though I guess Anaconda probably has more:

http://www.pyzo.org/packages.html

Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Paul Hobson-2
In reply to this post by Travis Oliphant-6



On Sat, Jun 29, 2013 at 5:01 PM, Travis Oliphant <[hidden email]> wrote:

Anaconda makes it easy to create an environment with Python 3 packages.   Many are already available.  

Try

conda create -n py33 python=3.3 numpy

look at anaconda packages to see all the available free binaries for python 3.3


At the risk of high jacking the thread, I'll ask you, Travis:
I just created oa Python 3.3 environment on my Windows work machine. Worked wonderfully and it's a testament to how hard y'all are working over at Continuum. Thanks for all of that. I point all our new staff to your Python 2.7 distro. However, the 3.3 environment is not in the registry, so if there's a package (particularly pyodbc in this case) that I *need* for work, I can't install it from the Golhke collection. 

Any plan to be able to register different python environments in Windows?

Thanks,
-paul  


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Paul Hobson-2
In reply to this post by Thomas Kluyver-2
The main road block holding me back, just as a user in the environmental consulting world, is the shapely/decartes combo. I believe that I only have two project relying on them, and I could probably work around that in some way, but I have a notebook in "production" for one project and no budget to make that switch.

Luckily though, that's on a separate machine from my main working environment, so I actually plan on making the switch soon (I've installed everything so far).



On Thu, Jun 27, 2013 at 5:22 PM, Thomas Kluyver <[hidden email]> wrote:
At a conversation over lunch here at the SciPy conference, a few of us mentioned that we're starting to use Python 3 in earnest for our work.

For new users, the choice of two major Python versions is confusing and offputting, and we're not going to completely get rid of that confusion until we can simply point new users to Python 3. Most of our introductions, like the SciPy stack install page, point to Python 2 because of the ecosystem, but more and more packages now support Python 3, and we're reaching the point where we could reasonably recommend Python 3 for new users.

The aim of this post is to get an overview of where the ecosystem is with:
- What packages don't yet support Python 3, or are still too unstable?
- How important are each of those: how widely relevant are they, and are substitutes available?
- What other conditions need to be met to recommend Python 3? E.g. Scientific Python distros, Linux distro packaging, documentation, etc.

Thanks,
Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Thomas Kluyver-2
On 30 June 2013 15:11, Paul Hobson <[hidden email]> wrote:
The main road block holding me back, just as a user in the environmental consulting world, is the shapely/decartes combo. I believe that I only have two project relying on them, and I could probably work around that in some way, but I have a notebook in "production" for one project and no budget to make that switch.

Thanks. I've started an Etherpad to track the key points of this discussion - feel free to add to it:

https://etherpad.mozilla.org/JdAHGQihei

Best wishes,
Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

ralfgommers
In reply to this post by Thomas Kluyver-2



On Sun, Jun 30, 2013 at 12:14 AM, Thomas Kluyver <[hidden email]> wrote:

On 29 June 2013 22:47, Ralf Gommers <[hidden email]> wrote:
Before recommending Python 3.x over 2.x I think it's important to not only have the very latest release or master branch of projects support 3.x, but at least 1 or 2 more versions. Reason: a lot of users (I suspect the majority) will not be able to freely upgrade to the latest version of projects.

I certainly wouldn't count a project as having Py3 support until there's a released version. But if they can't upgrade to the latest version, chances are that they also don't have a choice between Py2 and Py3, so our recommendation doesn't matter much to them. In that case, the recommendation is targeting the sysadmin who will be deciding what to install next year.

That's not quite what I meant. Even on a work pc on which I don't have admin rights I will be able to install Anaconda or another distribution. All the basic examples in Python and numpy/scipy docs will work. But I don't work in a vacuum, so I'll find out at some later stage that some code that my co-workers wrote depends on version (current minus 2) of some package that only supports 3.x in version (current). This should be the exception and not the norm before recommending 3.x imho.

Also, if many of the active developers haven't yet moved to 3.x (and yes that includes me) then it's most definitely too early to recommend said move to people who aren't very familiar with Python yet.

Cheers,
Ralf


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Hans-Martin v. Gaudecker-2
In reply to this post by ralfgommers
I have been largely working on Python 3 for two years now. I figured that starting long-term-projects with Python 2 was not worth it anymore. After I got some of those going, I gradually ported most of my other projects. Overall it has worked well for me -- with some packages I had to dig deeper into compiling things than I would have liked to, but that seems to be over now as I am not using many exotic things (mostly NumPy, SciPy, Matplotlib, Pandas, Statsmodels, some database stuff).

I am currently teaching a software-carpentry-inspired course to about 30 economics MSc students and everybody uses an Anaconda Python 3.3 environment. Again, this has worked very well. The list of included packages is probably a bit too short (especially when compared to Anaconda Python 2.7) as to recommend it by default to newcomers. Maybe this discussion will help to have the gap closed (even) faster.

On 01.07.13 19:00, [hidden email] wrote:
   1. Re: SciPy ecosystem and Python 3 (Ralf Gommers)

I will be able to install Anaconda or another distribution.
All the basic examples in Python and numpy/scipy docs will work. But I
don't work in a vacuum, so I'll find out at some later stage that some code
that my co-workers wrote depends on version (current minus 2) of some
package that only supports 3.x in version (current). This should be the
exception and not the norm before recommending 3.x imho.
I tend to disagree. If I arrive as a newcomer in a work environment where everybody else uses Python 2, I listen to my co-workers, use that, and don't care much about what the website recommends. The website is probably more relevant for people where a vacuum describes the environment fairly well.
Also, if many of the active developers haven't yet moved to 3.x (and yes
that includes me) then it's most definitely too early to recommend said
move to people who aren't very familiar with Python yet.
I would argue the other way round: Using Python 3 straight away will avoid a move for newcomers altogether. For long-time Python 2 users, the switching costs are particularly large -- which may explain the reluctance of many developers. But for newcomers, these costs could be avoided entirely (yes, they can be pretty small using 2to3 -- but that's not something one would want to explain to a newcomer in the first few hours, see Thomas' original post).

My 2c,
Hans-Martin


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Thomas Kluyver-2
In reply to this post by ralfgommers
On 30 June 2013 20:04, Ralf Gommers <[hidden email]> wrote:
That's not quite what I meant. Even on a work pc on which I don't have admin rights I will be able to install Anaconda or another distribution. All the basic examples in Python and numpy/scipy docs will work. But I don't work in a vacuum, so I'll find out at some later stage that some code that my co-workers wrote depends on version (current minus 2) of some package that only supports 3.x in version (current). This should be the exception and not the norm before recommending 3.x imho.

Again, though, I think your coworkers are more likely to have written code which expects Python 2, than Python-3-compatible code which relies on an older version of a particular library. But that's a chicken and egg problem, and if we always pointed newcomers at the option most likely to preserve compatibility with prior code, then we'd never have started using Python at all.

Hans also has a good point: if you're working in a group with a Python codebase, they should show you how to set up the preferred environment for that. I also imagine that we'll need to maintain a warning for some time after we start recommending Python 3, along the lines of "If you find code that doesn't work, it might be that it was never updated to run on Python 3." That's not ideal, but I think being able to point newcomers at the 'latest and greatest' version by default will still be an important improvement.

To my mind, the crucial prerequisite is getting robust Python 3 support in the packages that we (the open source SciPy community) develop. Someone has added quite a long list to the Etherpad (Thanks!), but some of them seem quite specialist, e.g. I've never even heard of Gamera or kwant before. I don't think we should hold the recommendation on every specific package that we can find, so the question is which of those packages are important.

Obviously that's somewhat subjective, so here's a couple of possible criteria to debate: A project is 'important' if
- It's relevant outside one specific field of study (i.e. we wouldn't block the general recommendation on a package specific to, say, quantum physics), and
- It's recommended by blog posts/tutorials/textbooks independent of the project and its main authors.

Thanks,
Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

josef.pktd
On Mon, Jul 1, 2013 at 7:55 PM, Thomas Kluyver <[hidden email]> wrote:

> On 30 June 2013 20:04, Ralf Gommers <[hidden email]> wrote:
>>
>> That's not quite what I meant. Even on a work pc on which I don't have
>> admin rights I will be able to install Anaconda or another distribution. All
>> the basic examples in Python and numpy/scipy docs will work. But I don't
>> work in a vacuum, so I'll find out at some later stage that some code that
>> my co-workers wrote depends on version (current minus 2) of some package
>> that only supports 3.x in version (current). This should be the exception
>> and not the norm before recommending 3.x imho.
>
>
> Again, though, I think your coworkers are more likely to have written code
> which expects Python 2, than Python-3-compatible code which relies on an
> older version of a particular library. But that's a chicken and egg problem,
> and if we always pointed newcomers at the option most likely to preserve
> compatibility with prior code, then we'd never have started using Python at
> all.
>
> Hans also has a good point: if you're working in a group with a Python
> codebase, they should show you how to set up the preferred environment for
> that. I also imagine that we'll need to maintain a warning for some time
> after we start recommending Python 3, along the lines of "If you find code
> that doesn't work, it might be that it was never updated to run on Python
> 3." That's not ideal, but I think being able to point newcomers at the
> 'latest and greatest' version by default will still be an important
> improvement.
>
> To my mind, the crucial prerequisite is getting robust Python 3 support in
> the packages that we (the open source SciPy community) develop. Someone has
> added quite a long list to the Etherpad (Thanks!), but some of them seem
> quite specialist, e.g. I've never even heard of Gamera or kwant before. I
> don't think we should hold the recommendation on every specific package that
> we can find, so the question is which of those packages are important.
>
> Obviously that's somewhat subjective, so here's a couple of possible
> criteria to debate: A project is 'important' if
> - It's relevant outside one specific field of study (i.e. we wouldn't block
> the general recommendation on a package specific to, say, quantum physics),
> and
> - It's recommended by blog posts/tutorials/textbooks independent of the
> project and its main authors.

I think it would be better to have a central list where
users/developers can add information about field specific packages. It
won't help those users if the scientific python core is available but
some crucial packages in their field are not available on python 3.

And not all fields have big communities that can do it on their own.

Josef


>
> Here's the pad again: https://etherpad.mozilla.org/JdAHGQihei
>
> Thanks,
> Thomas
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

ralfgommers
In reply to this post by Thomas Kluyver-2



On Tue, Jul 2, 2013 at 1:55 AM, Thomas Kluyver <[hidden email]> wrote:
On 30 June 2013 20:04, Ralf Gommers <[hidden email]> wrote:
That's not quite what I meant. Even on a work pc on which I don't have admin rights I will be able to install Anaconda or another distribution. All the basic examples in Python and numpy/scipy docs will work. But I don't work in a vacuum, so I'll find out at some later stage that some code that my co-workers wrote depends on version (current minus 2) of some package that only supports 3.x in version (current). This should be the exception and not the norm before recommending 3.x imho.

Again, though, I think your coworkers are more likely to have written code which expects Python 2, than Python-3-compatible code which relies on an older version of a particular library. But that's a chicken and egg problem, and if we always pointed newcomers at the option most likely to preserve compatibility with prior code, then we'd never have started using Python at all.

I understand it's a chicken and egg problem, but newcomers are not the right group to solve that. You want to give them the recommendation that helps them get started with the least amount of trouble. One bad experience (and having to do some serious debugging or even downgrade to 2.x is bad) can be enough to chase them back to Matlab.

We'll get there eventually, but only when a good portion (say 50%) of existing users and devs have moved.
 
Hans also has a good point: if you're working in a group with a Python codebase, they should show you how to set up the preferred environment for that.

If only the real world was that simple:)
 
I also imagine that we'll need to maintain a warning for some time after we start recommending Python 3, along the lines of "If you find code that doesn't work, it might be that it was never updated to run on Python 3."

No no no. That's a terrible sentence to write. Imagine you reading that if you move to new language X.

That's not ideal, but I think being able to point newcomers at the 'latest and greatest' version by default will still be an important improvement.

Please keep in mind that it's much more important to you, as an active dev who cares about 3.x adoption, then to them. All newcomers are getting for now is some compatibility issues and strings they don't understand. On the upside they don't have to move a few years later, but the business case is thin.

Cheers,
Ralf

P.S. I do agree with you on where we need to go, there's just no need to be in a hurry imho


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Nathaniel Polish
I am mostly a lurker here but as a practicing computer scientist and
engineer, I thought I'd offer my two cents.

We face this issue when getting into any system that is new to us.  Which
version of unix?  C, C++, Java, PHP, etc...All have their issues with
respect to versions.  All such systems have life cycles.  Its always best
to get your work done within a single product cycle.  Shifting in the
middle of project is usually deadly.

Anyone showing up to numpy etc now to get specific work done should really
be directed to 2.7.  That's where the maturity is.  3 is where the world is
going but the time scale is uncertain.  Pushing new users to 3 just to get
a community is a bad idea.  The experienced folks should be the ones
establishing the new version.  I know that everyone is trying...

I faced this two years ago when I starting working with numpy.  It was
confusing.  However I quickly realized as a CS person what was going on.
2.7 was solid and mature.  Errors that I found could almost always be
assumed to be my fault.  Everything I was doing could be assumed to have
been tried by someone else before.  Version 3 had none of those attributes
so I stuck to 2.7.  This is kind of the way that I feel about C and Unix.
Solid and mature.  Nothing much new.

For what its worth, if the community does not make the switch to version 3
reasonably soon, it will risk losing people to other systems that ARE
successfully moving through new versions.  Its hard with such a far flung
community but that's what we are.

There, I will now retreat to the shadows from whence I came.  You folks do
amazing work and I am mostly in awe of its quality.



--On Tuesday, July 02, 2013 8:47 AM +0200 Ralf Gommers
<[hidden email]> wrote:

>
>
>
>
>
>
> On Tue, Jul 2, 2013 at 1:55 AM, Thomas Kluyver <[hidden email]> wrote:
>
>
>
>
>
> On 30 June 2013 20:04, Ralf Gommers <[hidden email]> wrote:
>
> That's not quite what I meant. Even on a work pc on which I don't have
> admin rights I will be able to install Anaconda or another distribution.
> All the basic examples in Python and numpy/scipy docs will work. But I
> don't work in a vacuum, so I'll find out at some later stage that some
> code that my co-workers wrote depends on version (current minus 2) of
> some package that only supports 3.x in version (current). This should be
> the exception and not the norm before recommending 3.x imho.
>
>
>
> Again, though, I think your coworkers are more likely to have written
> code which expects Python 2, than Python-3-compatible code which relies
> on an older version of a particular library. But that's a chicken and egg
> problem, and if we always pointed newcomers at the option most likely to
> preserve compatibility with prior code, then we'd never have started
> using Python at all.
>
>
>
>
> I understand it's a chicken and egg problem, but newcomers are not the
> right group to solve that. You want to give them the recommendation that
> helps them get started with the least amount of trouble. One bad
> experience (and having to do some serious debugging or even downgrade to
> 2.x is bad) can be enough to chase them back to Matlab.
>
>
> We'll get there eventually, but only when a good portion (say 50%) of
> existing users and devs have moved.
>
>  
>
>
>
>
> Hans also has a good point: if you're working in a group with a Python
> codebase, they should show you how to set up the preferred environment
> for that.
>
>
>
>
> If only the real world was that simple:)
>  
>
>
>
> I also imagine that we'll need to maintain a warning for some time after
> we start recommending Python 3, along the lines of "If you find code that
> doesn't work, it might be that it was never updated to run on Python 3."
>
>
>
>
> No no no. That's a terrible sentence to write. Imagine you reading that
> if you move to new language X.
>
>
>
>
>  That's not ideal, but I think being able to point newcomers at the
> 'latest and greatest' version by default will still be an important
> improvement.
>
>
>
>
> Please keep in mind that it's much more important to you, as an active
> dev who cares about 3.x adoption, then to them. All newcomers are getting
> for now is some compatibility issues and strings they don't understand.
> On the upside they don't have to move a few years later, but the business
> case is thin.
>
>
> Cheers,
> Ralf
>
>
> P.S. I do agree with you on where we need to go, there's just no need to
> be in a hurry imho
>


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Thomas Kluyver-2
In reply to this post by ralfgommers
On 2 July 2013 07:47, Ralf Gommers <[hidden email]> wrote:
Please keep in mind that it's much more important to you, as an active dev who cares about 3.x adoption, then to them. All newcomers are getting for now is some compatibility issues and strings they don't understand. On the upside they don't have to move a few years later, but the business case is thin.

I'm not just doing this to cheerlead Python 3 adoption. Many of us have seen newcomers being confused by the split. I don't have references handy, but I've heard about courses that have asked people to preinstall Python, and despite careful instructions, people have turned up with a mixture of Python 2 and Python 3, which then wastes valuable time while everyone gets to the same starting point. Discussion sites see regular 'should I use 2 or 3' threads. And it's easy to imagine potential users who're evaluating Python against alternative solutions, and get put off by the 2/3 split, though we probably don't hear from them.

Again, I'm not saying that we should promote Python 3 now - there's still some way to go. I'm trying to define how far it is, and how we can measure it. You suggest we should wait until maybe 50% of existing users and devs have switched. I wouldn't wait that long, because I think it's fine for new users to change quicker than old users, but this is the sort of criterion I want to discuss.

How would we go about estimating how many users/devs use Python 3? It can seem like no-one is, but my conversations at SciPy, and bug reports on IPython, suggest that that's not entirely true.

Christoph (& Josef said something similar)
> I think a webpage summarising Python version compatibility would be a great resource.

That's an interesting idea. There are already sites out there which list Python 3 support for PyPI packages, but there's certainly room for something more detailed. Specifically, it could know about:
- Which minor Python versions does X support (e.g. 2.6, 2.7, 3.3)
- How has this support changed in recent releases of X
- Is X version y packaged for Python version z in Debian/Macports/Anaconda/etc.

Is anybody interested in creating this and keeping it updated? We could use PyPI classifiers as a starting point, but there would need to be extra information layered on top of that.

However, we shouldn't overstate the importance of this for newcomers: you know which field you're working in, but you often don't know which packages you're going to need until you've already written quite a bit of code. So you can't just look up the packages you'll need before you start with SciPy.

Nathaniel:
> I faced this two years ago when I starting working with numpy.  It was
> confusing.  However I quickly realized as a CS person what was going on.
> 2.7 was solid and mature.

Precisely. However, many of our target audience aren't CS people, and will only see that they're being asked to use an 'old' version for some reason.

Also, the Py3 ecosystem is much more mature than it was two years ago. At that time, it was easy to point to inarguably important packages like matplotlib as a reason to start with Python 2. Today, that's still possible in specific fields, but it's increasingly hard to find Python-2-only packages that new SciPy users in general are likely to need.

Best wishes,
Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

Matthew Brett
Hi,

On Tue, Jul 2, 2013 at 4:00 PM, Thomas Kluyver <[hidden email]> wrote:

> On 2 July 2013 07:47, Ralf Gommers <[hidden email]> wrote:
>>
>> Please keep in mind that it's much more important to you, as an active dev
>> who cares about 3.x adoption, then to them. All newcomers are getting for
>> now is some compatibility issues and strings they don't understand. On the
>> upside they don't have to move a few years later, but the business case is
>> thin.
>
>
> I'm not just doing this to cheerlead Python 3 adoption. Many of us have seen
> newcomers being confused by the split. I don't have references handy, but
> I've heard about courses that have asked people to preinstall Python, and
> despite careful instructions, people have turned up with a mixture of Python
> 2 and Python 3, which then wastes valuable time while everyone gets to the
> same starting point. Discussion sites see regular 'should I use 2 or 3'
> threads. And it's easy to imagine potential users who're evaluating Python
> against alternative solutions, and get put off by the 2/3 split, though we
> probably don't hear from them.

Agreeing with Thomas:

Most of us when starting with a new software stack, look for the
latest version.  I guess this is because it's fun to use the latest
stuff, and because it's annoying learning habits for stuff that will
soon be deprecated or raise an error.

We could explain why that should not be the case for scientific
python, but I imagine that new users will be a little puzzled and
maybe worried that we should be holding to an old version so long
after the release of the new.

I do believe we should have slight bias towards 3 rather 2 for the
sake of the health of the overall python ecosystem, which will have to
move.  I don't know how we would know when we should 'recommend' 3
though.

Best,

Matthew
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: SciPy ecosystem and Python 3

josef.pktd
In reply to this post by Thomas Kluyver-2
On Tue, Jul 2, 2013 at 11:00 AM, Thomas Kluyver <[hidden email]> wrote:

> On 2 July 2013 07:47, Ralf Gommers <[hidden email]> wrote:
>>
>> Please keep in mind that it's much more important to you, as an active dev
>> who cares about 3.x adoption, then to them. All newcomers are getting for
>> now is some compatibility issues and strings they don't understand. On the
>> upside they don't have to move a few years later, but the business case is
>> thin.
>
>
> I'm not just doing this to cheerlead Python 3 adoption. Many of us have seen
> newcomers being confused by the split. I don't have references handy, but
> I've heard about courses that have asked people to preinstall Python, and
> despite careful instructions, people have turned up with a mixture of Python
> 2 and Python 3, which then wastes valuable time while everyone gets to the
> same starting point. Discussion sites see regular 'should I use 2 or 3'
> threads. And it's easy to imagine potential users who're evaluating Python
> against alternative solutions, and get put off by the 2/3 split, though we
> probably don't hear from them.
>
> Again, I'm not saying that we should promote Python 3 now - there's still
> some way to go. I'm trying to define how far it is, and how we can measure
> it. You suggest we should wait until maybe 50% of existing users and devs
> have switched. I wouldn't wait that long, because I think it's fine for new
> users to change quicker than old users, but this is the sort of criterion I
> want to discuss.
>
> How would we go about estimating how many users/devs use Python 3? It can
> seem like no-one is, but my conversations at SciPy, and bug reports on
> IPython, suggest that that's not entirely true.
>
> Christoph (& Josef said something similar)
>> I think a webpage summarising Python version compatibility would be a
>> great resource.
>
> That's an interesting idea. There are already sites out there which list
> Python 3 support for PyPI packages, but there's certainly room for something
> more detailed. Specifically, it could know about:
> - Which minor Python versions does X support (e.g. 2.6, 2.7, 3.3)
> - How has this support changed in recent releases of X
> - Is X version y packaged for Python version z in
> Debian/Macports/Anaconda/etc.
>
> Is anybody interested in creating this and keeping it updated? We could use
> PyPI classifiers as a starting point, but there would need to be extra
> information layered on top of that.
>
> However, we shouldn't overstate the importance of this for newcomers: you
> know which field you're working in, but you often don't know which packages
> you're going to need until you've already written quite a bit of code. So
> you can't just look up the packages you'll need before you start with SciPy.

We have a http://scipy.org/topical-software.html which is partially
maintained, and maintained almost only by the community. A similar
information for the python 3 status would be a good starting point for
users to decide on the python version.

>
> Nathaniel:
>
>> I faced this two years ago when I starting working with numpy.  It was
>> confusing.  However I quickly realized as a CS person what was going on.
>> 2.7 was solid and mature.
>
> Precisely. However, many of our target audience aren't CS people, and will
> only see that they're being asked to use an 'old' version for some reason.
>
> Also, the Py3 ecosystem is much more mature than it was two years ago. At
> that time, it was easy to point to inarguably important packages like
> matplotlib as a reason to start with Python 2. Today, that's still possible
> in specific fields, but it's increasingly hard to find Python-2-only
> packages that new SciPy users in general are likely to need.

(started to write this in response to Nathaniel but now fits better here)

numpy, scipy, pandas, statsmodels have been available for python 3 for
more than 2 years now. We usually get compatibility with new python 3
versions as soon as they come out.

(aside: python 3 only http://biogeme.epfl.ch/doc/install.html for
transportation modelling )

I think the recommendation should depend on the setting a user is in.
I'm mostly with Hans-Martin that students that don't move immediately
into an established production setting, or new users without an
established 2.x peer group should start with python 3, especially if
it only takes another year until the stragglers are also available on
python 3. (spyder is on it's way)

my impression from the statsmodels conversion two years ago:
numerical code, all our base algorithms, required almost no changes in
moving to python 3, the main adjustments were in input and output. So,
I think, the numerical code is essentially as reliable on python 3 as
on python 2.
(of course this build largely on top of the changes that numpy and scipy made.)

About developers:
I'm working at the tail end of the supported dependencies of
statsmodels, so I can catch backwards compatibility problems.
But as a user, I would prefer to work with the "latest and greatest"
and take advantage of new features instead of lagging several years
behind.

Cheers,

Josef

>
> Best wishes,
> Thomas
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
12