[SciPy-User] Alternative for scipy.sparse.sparsetools for use from outside of scipy

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] Alternative for scipy.sparse.sparsetools for use from outside of scipy

Lev Konstantinovskiy
Hi,

Getting deprecation warning for sparsetools. Is there an alternative to
switch to?

The use is sparsetools.csc_matvecs in gensim
https://github.com/piskvorky/gensim/blob/9a1c2c954e2f72213023fc01f0e33306001e
303f/gensim/models/lsimodel.py

warning on import:

gensim/home/ubuntu/.vew/ds26/lib/python2.6/site-
packages/numpy/lib/utils.py:95: DeprecationWarning:
`scipy.sparse.sparsetools` is deprecated!
scipy.sparse.sparsetools is a private module for scipy.sparse, and should not
be used.
  warnings.warn(depdoc, DeprecationWarning)

Thanks


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Alternative for scipy.sparse.sparsetools for use from outside of scipy

Pauli Virtanen-3
Lev Konstantinovskiy <lev.konst <at> gmail.com> writes:
> Getting deprecation warning for sparsetools. Is there an alternative to
> switch to?
>
> The use is sparsetools.csc_matvecs in gensim
> https://github.com/piskvorky/gensim/blob/9a1c2c954e2f72213023fc01f0e33306001e
> 303f/gensim/models/lsimodel.py

If I understand correctly, these are use cases that can be expressed
in terms of usual sparse matrix operations,

y = corpus * o
y += corpus * chunk

but you are using the internal sparsetools routines instead,
because of performance reasons? Is the performance difference
big in this case? Is the issue that you want in-place sparse AXPY,
or is it due to dealing with small matrices and avoiding
overheads?

There's currently no sparse axpy available in Scipy.
There probably should be though.

Gensim is not a pure-Python module, so one relatively straightforward
possibility is to just bundle a copy of the current sparsetools
module (or just the one routine you need) with it. There's no SWIG
nowadays involved, and it's independent of the rest of Scipy,
so it's probably doable.

--
Pauli Virtanen

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Alternative for scipy.sparse.sparsetools for use from outside of scipy

Lev Konstantinovskiy
Thanks for your reply. Raised a feature request for sparse in-place AXPY
https://github.com/scipy/scipy/issues/5348

The reason for using this routine is to save memory. Using
sparse.linalg.matmat would double the memory in
 y = y + chunk * o
to create a temporary chunk*o. But it seems the only way to go when
sparsetools becomes deprecated.

On Tue, Oct 6, 2015 at 8:28 PM, Pauli Virtanen <[hidden email]> wrote:

> Lev Konstantinovskiy <lev.konst <at> gmail.com> writes:
>> Getting deprecation warning for sparsetools. Is there an alternative to
>> switch to?
>>
>> The use is sparsetools.csc_matvecs in gensim
>> https://github.com/piskvorky/gensim/blob/9a1c2c954e2f72213023fc01f0e33306001e
>> 303f/gensim/models/lsimodel.py
>
> If I understand correctly, these are use cases that can be expressed
> in terms of usual sparse matrix operations,
>
> y = corpus * o
> y += corpus * chunk
>
> but you are using the internal sparsetools routines instead,
> because of performance reasons? Is the performance difference
> big in this case? Is the issue that you want in-place sparse AXPY,
> or is it due to dealing with small matrices and avoiding
> overheads?
>
> There's currently no sparse axpy available in Scipy.
> There probably should be though.
>
> Gensim is not a pure-Python module, so one relatively straightforward
> possibility is to just bundle a copy of the current sparsetools
> module (or just the one routine you need) with it. There's no SWIG
> nowadays involved, and it's independent of the rest of Scipy,
> so it's probably doable.
>
> --
> Pauli Virtanen
>
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> https://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/scipy-user