Central File Exchange for SciPy

classic Classic list List threaded Threaded
90 messages Options
12345
O-8
Reply | Threaded
Open this post in threaded view
|

Central File Exchange for SciPy

O-8

Hi everyone,

I'm a recent convert from MatLab.

One thing I miss is the Central File Exchange.  Are there any plans
to set up a site like this for our community?  It occurs to me this could
dramatically strengthen our user base.  And by a "Central File Exchange"
I mean something far simpler and less formal than "SciKits", where users
can just post their code with some information about how it works.

Just a thought.

Cheers,

O



_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

josef.pktd
On Sun, Oct 24, 2010 at 10:43 AM, O <[hidden email]> wrote:

>
> Hi everyone,
>
> I'm a recent convert from MatLab.
>
> One thing I miss is the Central File Exchange.  Are there any plans
> to set up a site like this for our community?  It occurs to me this could
> dramatically strengthen our user base.  And by a "Central File Exchange"
> I mean something far simpler and less formal than "SciKits", where users
> can just post their code with some information about how it works.

There are methods available to publish your code more easily than a
scikits, if it's just a single module than the cookbook is a good
location, http://www.scipy.org/Cookbook . For anything larger, setting
up a simple python package for pypi is relatively easy, e.g.
numdifftools which is a translation of the matlab fileexchange program
coauthored jointly with the file exchange author.

The main thing I'm missing compared to the file exchange is the
comment and starring system, which reduces the time to check out a new
package a lot. Also compared to matlab developers, python developers
are often keeping public source control repositories which makes
finding "recipes" easier, but again finding something specific is a
bit of a random search. A good but incomplete overview is in
http://www.scipy.org/Topical_Software

Maybe we could extend the purpose of ask.scipy.org to a package
review/commenting and package "advertising". But I haven't found a
search button on it yet.

Josef


>
> Just a thought.
>
> Cheers,
>
> O
>
>
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

denis-bz-gg
> On Sun, Oct 24, 2010 at 10:43 AM, O <[hidden email]> wrote:
> > I'm a recent convert from MatLab.
> > One thing I miss is the Central File Exchange.  Are there any plans
...

O,
  can you describe a bit which parts of Matlab Central you want most ?
As Josef says, Scipy uses mail forums, ask.scipy.org,
stackoverflow ...
to answer questions pretty fast. Do you / do other users want

- package reviews / comments / advertising --
    for new users, for experts ? Examples from Matlab ?

- overviews of major areas, along the lines of Wikipedia articles
    with links to detailed doc and recipes ?

I believe that Scipy experts get more points for new stuff
and for answering questions than they would for either of these.
Matlab has a different reward system:

    "I believe in Art, but my manager believes in money,
    and who am I to argue with such a baboon ?"
        -- Groucho Marx

cheers
  -- denis
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Almar Klein
I agree with the OP. The Matlab file exchange is a great tool for developers to easily publish small (but also larger) pieces of code, and for people to search for particular code.

As to what features I think such a system should have: It should be a repository capable of storing thousands of entries, which should be indexed, and categorized so that users can easily find code they are looking for. A review system and comments would also be nice.

One `problem´ is that many Python developers that have a great tool, publish it as an open source project on googlecode for example. Maybe such projects could be entered in the database as well, with a reference to the googlecode website for the code itself.

I think it is a great idea, as it would help unite the Python scientific community. One (maybe the only) disadvantage I found when converting from Matlab to the Holy Language, is that the Python world seems a bit fragmented; you have to download Python from www.python.org, then numpy and scipy from scipy.org, etc. A repository of the likes of the Matlab file exchange would put at least all non standard code in a single place, which would be a big advantage.

But who's going to set up and maintain such a big project, and whos going to pay for the server?

Cheers,
  Almar


On 29 October 2010 18:15, denis <[hidden email]> wrote:
> On Sun, Oct 24, 2010 at 10:43 AM, O <[hidden email]> wrote:
> > I'm a recent convert from MatLab.
> > One thing I miss is the Central File Exchange.  Are there any plans
...

O,
 can you describe a bit which parts of Matlab Central you want most ?
As Josef says, Scipy uses mail forums, ask.scipy.org,
stackoverflow ...
to answer questions pretty fast. Do you / do other users want

- package reviews / comments / advertising --
   for new users, for experts ? Examples from Matlab ?

- overviews of major areas, along the lines of Wikipedia articles
   with links to detailed doc and recipes ?

I believe that Scipy experts get more points for new stuff
and for answering questions than they would for either of these.
Matlab has a different reward system:

   "I believe in Art, but my manager believes in money,
   and who am I to argue with such a baboon ?"
       -- Groucho Marx

cheers
 -- denis
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Alan G Isaac-2
On 10/29/2010 4:53 PM, Almar Klein wrote:
> The Matlab file exchange is a great tool for developers to easily publish small (but also larger) pieces of code

The Cookbook holds small pieces of code:
http://www.scipy.org/Cookbook

fwiw,
Alan Isaac

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Joshua Holbrook
I'm surprised nobody's mentioned pypi yet:  http://pypi.python.org/pypi

--Josh

On Fri, Oct 29, 2010 at 2:07 PM, Alan G Isaac <[hidden email]> wrote:

> On 10/29/2010 4:53 PM, Almar Klein wrote:
>> The Matlab file exchange is a great tool for developers to easily publish small (but also larger) pieces of code
>
> The Cookbook holds small pieces of code:
> http://www.scipy.org/Cookbook
>
> fwiw,
> Alan Isaac
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Almar Klein
In reply to this post by Alan G Isaac-2


On 30 October 2010 00:07, Alan G Isaac <[hidden email]> wrote:
On 10/29/2010 4:53 PM, Almar Klein wrote:
> The Matlab file exchange is a great tool for developers to easily publish small (but also larger) pieces of code

The Cookbook holds small pieces of code:

Yes, but it wouldn't really work if hundreds (or thousands) of people would submit pieces of code.

  Almar

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Gerrit Holl-2
On 30 October 2010 13:44, Almar Klein <[hidden email]> wrote:

> On 30 October 2010 00:07, Alan G Isaac <[hidden email]> wrote:
>> On 10/29/2010 4:53 PM, Almar Klein wrote:
>> > The Matlab file exchange is a great tool for developers to easily
>> > publish small (but also larger) pieces of code
>>
>> The Cookbook holds small pieces of code:
>> http://www.scipy.org/Cookbook
>
> Yes, but it wouldn't really work if hundreds (or thousands) of people would
> submit pieces of code.

Why not add it to scipy? Or if it doesn't fit put it somewhere and
link it from http://www.scipy.org/Topical_Software

Gerrit.
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Gael Varoquaux
On Sat, Oct 30, 2010 at 02:02:46PM +0200, Gerrit Holl wrote:
> >> The Cookbook holds small pieces of code:
> >> http://www.scipy.org/Cookbook

> > Yes, but it wouldn't really work if hundreds (or thousands) of people would
> > submit pieces of code.

> Why not add it to scipy?

Because code requires maintenance, releases, and quality assurance. If
thousands of people start pushing code in scipy, they need to help doing
both of these things.

The goal of a repo with no garanties like Matlab Central is to lower the
barrier to sharing code, but giving up on any garanties.

> Or if it doesn't fit put it somewhere and
> link it from http://www.scipy.org/Topical_Software

That means buiding software packages, which is also more work than simply
dumping code on a webpage.

Gaël
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Pauli Virtanen-3
In reply to this post by Gerrit Holl-2
Sat, 30 Oct 2010 14:02:46 +0200, Gerrit Holl wrote:
[clip]
>> Yes, but it wouldn't really work if hundreds (or thousands) of people
>> would submit pieces of code.
>
> Why not add it to scipy?

Because Scipy should only receive general-purpose and good-quality code,
and its release cycle is not that fast.

On purpose:

The scope of Scipy is mainly to contain "basic tools for numerical
scientific computation".

On quality:

What you typically have at first is "research quality code" --- it works
for your particular problem, but it might not do everything necessary,
may actually be a poor way to solve the problem, you are not 100% sure it
has no bugs, and you haven't tested it for other problems. Refining it
from this point onwards takes quite a bit of effort.

On speed:

You typically would like to publish your code now and not wait for a year
before it's out.

--
Pauli Virtanen

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Matthew Brett
In reply to this post by Gael Varoquaux
Hi,

>> Or if it doesn't fit put it somewhere and
>> link it from http://www.scipy.org/Topical_Software
>
> That means buiding software packages, which is also more work than simply
> dumping code on a webpage.

It's probably worth pointing out that most Matlab utilities are Matlab
only (no extensions), and the code dumped is usually just an archive
that you unpack somewhere and put on your Matlab path.

That's the rough equivalent of a python package that is pure python,
and for which the install method is copying or linking the <mypackage>
directory into some directory on your python path.

I can imagine something like a 'snippet' distribution format, which is
just a README file, and the <mypackage> directory.  Obviously if
someone wanted to be more pypi about the whole thing, that would be
easy too.

See y'all,

Matthew
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

John [H2O]
In reply to this post by Joshua Holbrook
+1 on pypi, but it needs some features added to it. Creating something
between Matlab file exchange and vim scripts would be ideal.

I've just checked out ask.scipy.org, but this doesn't seem what the OP
is after. Furthermore, is there really no search feature here??

I think what pypi needs to fill this void is some features such as:

1) a separation of simple scripts versus packages (so really we need
PyScript... also)

2) a good rating system

3) a good comment system

--john


On Sat, Oct 30, 2010 at 12:10 AM, Joshua Holbrook
<[hidden email]> wrote:

> I'm surprised nobody's mentioned pypi yet:  http://pypi.python.org/pypi
>
> --Josh
>
> On Fri, Oct 29, 2010 at 2:07 PM, Alan G Isaac <[hidden email]> wrote:
>> On 10/29/2010 4:53 PM, Almar Klein wrote:
>>> The Matlab file exchange is a great tool for developers to easily publish small (but also larger) pieces of code
>>
>> The Cookbook holds small pieces of code:
>> http://www.scipy.org/Cookbook
>>
>> fwiw,
>> Alan Isaac
>>
>> _______________________________________________
>> SciPy-User mailing list
>> [hidden email]
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>



--
Configuration
``````````````````````````
Plone 2.5.3-final,
CMF-1.6.4,
Zope (Zope 2.9.7-final, python 2.4.4, linux2),
Python 2.6
PIL 1.1.6
Mailman 2.1.9
Postfix 2.4.5
Procmail v3.22 2001/09/10
Basemap: 1.0
Matplotlib: 1.0.0
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

John [H2O]
In reply to this post by Matthew Brett
Speaking of pypi, does anyone know if there is a way to see where the
downloads came from as a package distributor. It would be interesting
information.

-john

On Sat, Oct 30, 2010 at 2:15 PM, Matthew Brett <[hidden email]> wrote:

> Hi,
>
>>> Or if it doesn't fit put it somewhere and
>>> link it from http://www.scipy.org/Topical_Software
>>
>> That means buiding software packages, which is also more work than simply
>> dumping code on a webpage.
>
> It's probably worth pointing out that most Matlab utilities are Matlab
> only (no extensions), and the code dumped is usually just an archive
> that you unpack somewhere and put on your Matlab path.
>
> That's the rough equivalent of a python package that is pure python,
> and for which the install method is copying or linking the <mypackage>
> directory into some directory on your python path.
>
> I can imagine something like a 'snippet' distribution format, which is
> just a README file, and the <mypackage> directory.  Obviously if
> someone wanted to be more pypi about the whole thing, that would be
> easy too.
>
> See y'all,
>
> Matthew
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

josef.pktd
In reply to this post by Gael Varoquaux
On Sat, Oct 30, 2010 at 8:07 AM, Gael Varoquaux
<[hidden email]> wrote:
> On Sat, Oct 30, 2010 at 02:02:46PM +0200, Gerrit Holl wrote:
>> >> The Cookbook holds small pieces of code:
>> >> http://www.scipy.org/Cookbook

there is also the python cookbook (the interface looks closer to
stackoverflow now)
http://code.activestate.com/recipes/tags/numeric/

>
>> > Yes, but it wouldn't really work if hundreds (or thousands) of people would
>> > submit pieces of code.
>
>> Why not add it to scipy?
>
> Because code requires maintenance, releases, and quality assurance. If
> thousands of people start pushing code in scipy, they need to help doing
> both of these things.
>
> The goal of a repo with no garanties like Matlab Central is to lower the
> barrier to sharing code, but giving up on any garanties.

The big advantage in my view of the matlab file exchange is the
ability to comment and rate an existing package and fork it if it
looks like it can be improved with attribution link and has inspired
links. And given that it is all (new code) clearly defined as BSD, it
is safe to do so.

This improves the quality control problem for the user quite a bit.

The problem with pypi and "Topical Software", and as seen in the
question on neural networks, is, for example, that dead and active
projects are indistinguishable without finding the source repository
and checking the updates.

Without user contributed commenting it is a lot of work to maintain a
list, see the (non)speed in cleaning up dead links on the Topical
page.

Josef

>
>> Or if it doesn't fit put it somewhere and
>> link it from http://www.scipy.org/Topical_Software
>
> That means buiding software packages, which is also more work than simply
> dumping code on a webpage.
>
> Gaël
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Almar Klein
In reply to this post by John [H2O]


On 30 October 2010 00:10, Joshua Holbrook <[hidden email]> wrote:
I'm surprised nobody's mentioned pypi yet:  http://pypi.python.org/pypi

You're right. Pypi already has quite a few of the required features.
But still, for some reason I cannot put my finger on, Matlab central feels nicer.


On 30 October 2010 14:15, John <[hidden email]> wrote:

I think what pypi needs to fill this void is some features such as:

1) a separation of simple scripts versus packages (so really we need
PyScript... also)

2) a good rating system

3) a good comment system

These features might indeed be improved a bit.

  Almar



_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

josef.pktd
In reply to this post by Matthew Brett
On Sat, Oct 30, 2010 at 8:15 AM, Matthew Brett <[hidden email]> wrote:

> Hi,
>
>>> Or if it doesn't fit put it somewhere and
>>> link it from http://www.scipy.org/Topical_Software
>>
>> That means buiding software packages, which is also more work than simply
>> dumping code on a webpage.
>
> It's probably worth pointing out that most Matlab utilities are Matlab
> only (no extensions), and the code dumped is usually just an archive
> that you unpack somewhere and put on your Matlab path.
>
> That's the rough equivalent of a python package that is pure python,
> and for which the install method is copying or linking the <mypackage>
> directory into some directory on your python path.
>
> I can imagine something like a 'snippet' distribution format, which is
> just a README file, and the <mypackage> directory.  Obviously if
> someone wanted to be more pypi about the whole thing, that would be
> easy too.

For plain python packages, "paster create" provides a full package
structure with just filling out a few questions

>paster create --list-templates
Available templates:
  basic_package:    A basic setuptools-enabled package
  complete:         Complete, documentable, testable Python project template
...

Just to be more pypi about it.
(The only explanation a very short google search provides, is how it
can be used for zope templates
http://plone.org/documentation/kb/use-paster)

Josef
>
> See y'all,
>
> Matthew
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

william ratcliff
If we could automate it, how much do you think the bandwidth/hosting costs would be per month?   Would it be restricted to just code (that is just text files, cutoff above a certain size)?  No bug tracking and a simple rating system for packages? A section for comments about a given package.  The submitter gives it up to 4 tags (for searching) and we start out with a given list of topics and let people additional ones later?  People register for an account (to reduce spam) or do we just use Openid or Openauth?     How do we deal with spam?  Do we allow people to sort packages by date?  Rating?  Would people want to use Django?   What would we call it?  I'd be willing to purchase a domain name and pay for hosting on webfaction to try it out.  If it gets too pricey then I may have to ask for help later. (I think we should avoid ads).    What would you guys like to call it?

PythonCentral (is that infringing?)?  ScipyExchange?    

If anyone wants to help mock up a prototype in Django, I have some time next week.  I have no design skills ;> 

Finally, licensing--I don't want to start a flame war or anything, but can we agree to make code on the site BSD, or should we allow the submitter to pick an open source license.  If so, do we follow googlecode for the choice of license?

One last question (sorry for so many),  given how many people already have nice projects on github, sourceforge, googlecode, etc., should we provide an option for people to simply link to their repository rather than provide us with a direct copy of the code?  Actually, one model could be that people host their code somewhere else and we merely provide an aggregation service so people can easily see what's out there in the scientific python universe and how the community has rated a given package.   That way, developers can keep their existing codebases without changing their workflow....

William

On Sat, Oct 30, 2010 at 9:39 AM, <[hidden email]> wrote:
On Sat, Oct 30, 2010 at 8:15 AM, Matthew Brett <[hidden email]> wrote:
> Hi,
>
>>> Or if it doesn't fit put it somewhere and
>>> link it from http://www.scipy.org/Topical_Software
>>
>> That means buiding software packages, which is also more work than simply
>> dumping code on a webpage.
>
> It's probably worth pointing out that most Matlab utilities are Matlab
> only (no extensions), and the code dumped is usually just an archive
> that you unpack somewhere and put on your Matlab path.
>
> That's the rough equivalent of a python package that is pure python,
> and for which the install method is copying or linking the <mypackage>
> directory into some directory on your python path.
>
> I can imagine something like a 'snippet' distribution format, which is
> just a README file, and the <mypackage> directory.  Obviously if
> someone wanted to be more pypi about the whole thing, that would be
> easy too.

For plain python packages, "paster create" provides a full package
structure with just filling out a few questions

>paster create --list-templates
Available templates:
 basic_package:    A basic setuptools-enabled package
 complete:         Complete, documentable, testable Python project template
...

Just to be more pypi about it.
(The only explanation a very short google search provides, is how it
can be used for zope templates
http://plone.org/documentation/kb/use-paster)

Josef
>
> See y'all,
>
> Matthew
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Pauli Virtanen-3
Sat, 30 Oct 2010 11:00:47 -0400, william ratcliff wrote:
> If we could automate it, how much do you think the bandwidth/hosting
> costs would be per month?

No idea. Probably the traffic wouldn't be too much, at least at first.

> No bug tracking and a simple rating system for packages?

Yes.

> A section for comments about a given package.

Perhaps with a possibility to up/downvote comments?

> The submitter gives it up to 4 tags (for searching) and we
> start out with a given list of topics and let people additional ones
> later?  

Yes. It might be useful to try to follow PyPi style classifiers here, and
extend them as needed.

> People register for an account (to reduce spam) or do we just use
> Openid or Openauth? How do we deal with spam?

Email verification on registration + spam flagging by users +
rel=nofollow in comments?

> Do we allow people to sort packages by date? Rating?  

Yes and yes.

> Would people want to use Django?  

Django will get the job done, and it's on the easier end of the spectrum
of Python web frameworks. I'd pick it.

> I'd be willing to purchase a domain name and pay for hosting on
> webfaction to try it out.  If it gets too pricey then I may have to
> ask for help later. (I think we should avoid ads).

One possibility might be to ask if Enthough would be interested in
sponsoring such a thing, and running it on the scipy.org servers. But
that's for later, when there's actually an something working to show.

> What would you guys like to call it?
> PythonCentral (is that infringing?)?  ScipyExchange?

Well, it might be worth to target it for the scientific audience, so the
name choice should be in accord. Also, I'd avoid clone-ish names.

> If anyone wants to help mock up a prototype in Django, I have some time
> next week.  I have no design skills ;>

I know some Django.

> Finally, licensing--I don't want to start a flame war or anything, but
> can we agree to make code on the site BSD, or should we allow the
> submitter to pick an open source license.  If so, do we follow
> googlecode for the choice of license?

I'd believe allowing the submitter to pick an open-source license for
bigger packages could be useful.

However, for code snippets we might want to enforce BSD.

> One last question (sorry for so many),  given how many people already
> have nice projects on github, sourceforge, googlecode, etc., should we
> provide an option for people to simply link to their repository rather
> than provide us with a direct copy of the code?  Actually, one model
> could be that people host their code somewhere else and we merely
> provide an aggregation service so people can easily see what's out there
> in the scientific python universe and how the community has rated a
> given package.   That way, developers can keep their existing codebases
> without changing their workflow....

Here, it would be best to not forget that we already have the scikits.*
namespace packages, and scikits.appspot.com. How that web app works, is
that people just upload a package named scikits.something on PyPi, and
the portal picks it up from there.

The new system should be a "spiritual successor" to scikits, with more
features etc., and a friendly hosting option for small snippets.

So yes, it should definitely allow externally hosted packages, especially
PyPi. Perhaps it would even be useful to automatically import science
packages (including scikits) from PyPi. The package entry should also be
usable only as an "advertisement" for a package, with the package itself
being hosted elsewhere. (Here, users should be able to flag broken links
etc.)

Another thing that should be considered: the system should enforce that
metadata is entered: package descriptions should be sufficiently
detailed, a suitable number of tags should be entered, etc.

--
Pauli Virtanen

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

Almar Klein
In reply to this post by william ratcliff


On 30 October 2010 17:00, william ratcliff <[hidden email]> wrote:
If we could automate it, how much do you think the bandwidth/hosting costs would be per month?   Would it be restricted to just code (that is just text files, cutoff above a certain size)?  No bug tracking and a simple rating system for packages? A section for comments about a given package.  The submitter gives it up to 4 tags (for searching) and we start out with a given list of topics and let people additional ones later?  People register for an account (to reduce spam) or do we just use Openid or Openauth?     How do we deal with spam?  Do we allow people to sort packages by date?  Rating?  Would people want to use Django?   What would we call it?  I'd be willing to purchase a domain name and pay for hosting on webfaction to try it out.  If it gets too pricey then I may have to ask for help later. (I think we should avoid ads).    What would you guys like to call it?

Woaw, I like your enthusiasm! However, let's first establish whether we should discard Pypi or if we can maybe make it suitable for our needs with a few changes (assuming that the rest of the Python community lets us make these changes).

One maybe-downside is that Pypi is for Python in general. Is this a problem, do we want something purely for science and engineering?



PythonCentral (is that infringing?)?  ScipyExchange?    

If we're doing this, I guess it'd be science focused, so I suggest a name with a reference to science of scipy.

 
If anyone wants to help mock up a prototype in Django, I have some time next week.  I have no design skills ;> 

Finally, licensing--I don't want to start a flame war or anything, but can we agree to make code on the site BSD, or should we allow the submitter to pick an open source license.  If so, do we follow googlecode for the choice of license?

Given that Python is mainly BSD oriented, I would vote for making all code hosted at the site BSD. Maybe that larger projects that are only referenced (as you also suggested) may choose their own license.

(I actually own two non-BSD projects because I did not fully understand the value/importance of the BSD license in the Python world. I was recently convinced by a wise man and will convert both my projects to BSD.)
 

One last question (sorry for so many),  given how many people already have nice projects on github, sourceforge, googlecode, etc., should we provide an option for people to simply link to their repository rather than provide us with a direct copy of the code?  Actually, one model could be that people host their code somewhere else and we merely provide an aggregation service so people can easily see what's out there in the scientific python universe and how the community has rated a given package.   That way, developers can keep their existing codebases without changing their workflow....

I definitely think this is a good idea. The site would then serve the role as the central place to search for scientific Python projects, without the need for people to host their projects at two locations.

  Almar

PS: While writing this, Pauli also sent his response. I'm happy to see that we agree on most topics :)


William


On Sat, Oct 30, 2010 at 9:39 AM, <[hidden email]> wrote:
On Sat, Oct 30, 2010 at 8:15 AM, Matthew Brett <[hidden email]> wrote:
> Hi
>
>>> Or if it doesn't fit put it somewhere and
>>> link it from http://www.scipy.org/Topical_Software
>>
>> That means buiding software packages, which is also more work than simply
>> dumping code on a webpage.
>
> It's probably worth pointing out that most Matlab utilities are Matlab
> only (no extensions), and the code dumped is usually just an archive
> that you unpack somewhere and put on your Matlab path.
>
> That's the rough equivalent of a python package that is pure python,
> and for which the install method is copying or linking the <mypackage>
> directory into some directory on your python path.
>
> I can imagine something like a 'snippet' distribution format, which is
> just a README file, and the <mypackage> directory.  Obviously if
> someone wanted to be more pypi about the whole thing, that would be
> easy too.

For plain python packages, "paster create" provides a full package
structure with just filling out a few questions

>paster create --list-templates
Available templates:
 basic_package:    A basic setuptools-enabled package
 complete:         Complete, documentable, testable Python project template
...

Just to be more pypi about it.
(The only explanation a very short google search provides, is how it
can be used for zope templates
http://plone.org/documentation/kb/use-paster)

Josef
>
> See y'all,
>
> Matthew
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
O-8
Reply | Threaded
Open this post in threaded view
|

Re: Central File Exchange for SciPy

O-8
In reply to this post by O-8
William,

Bravo! If you decide to follow through on this, I think it could be *huge*.  One other question is whether to do it for just scipy or python generally.  I think people really need a place to deposit snipits of useful code w/rating system and commentary.  If it takes off, I think it would attract lots more people to Python and scipy/numpy.

I agree with all of your suggestions.  Maybe contact python.org to see if you can get a link to it.  As for what to call it, "PythonExchange" is a third option. 

With respect to licensing, I'd allow the submitters to choose, and supply guidelines on the site about how to do this properly in their code (if they want to).

Allowing people to link to projects they have elsewhere is an excellent idea.

O (phaustus)









_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
12345