__eq__ for scipy.sparse not working?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

__eq__ for scipy.sparse not working?

Joseph Turian
I am having trouble determining equality of sparse matrices.
Consider this code snippet. Although the sparse matrices appear to be equal, z==y returns false (until I convert the matrices to dense matrices).
What is the problem with the equality test here?

Thanks,

  Joseph

=================

import scipy.sparse
import numpy

x = scipy.sparse.csc_matrix((500,3))
x[(10, 1)] = 1
x[(20, 2)] = 2
y = numpy.asarray([[1., 2], [3, 4], [2, 1]])

z = x.dot(y)
assert(z.shape == (500,2))

w = scipy.sparse.csc_matrix((500,2))
w[(10, 0)] = 3.
w[(20, 0)] = 4
w[(10, 1)] = 4
w[(20, 1)] = 2
assert(z.shape == w.shape)
assert(type(z) == type(w))
assert(z.dtype == w.dtype)
print z
print w
print "z:", z
print "w:", w
print "type(z) == type(w):", type(z) == type(w)
print "dtype(z) == dtype(w):", z.dtype == w.dtype

print cmp(z, w)
print "Sparse z and w are equal:", z == w
z = z.todense()
w = w.todense()
print "Dense z and w are equal:", (z == w).all() == True


_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: __eq__ for scipy.sparse not working?

Nathan Bell-4
On Mon, Apr 14, 2008 at 4:45 PM, Joseph Turian <[hidden email]> wrote:
> I am having trouble determining equality of sparse matrices.
> Consider this code snippet. Although the sparse matrices appear to be equal,
> z==y returns false (until I convert the matrices to dense matrices).
> What is the problem with the equality test here?

Sparse matrices don't currently support that functionality.  A
workaround could be abs(A-B).nnz == 0

--
Nathan Bell [hidden email]
http://graphics.cs.uiuc.edu/~wnbell/
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: __eq__ for scipy.sparse not working?

Joseph Turian
Is there a reason that sparse matrices don't support this functionality?
What is actually happening when I try equality testing for A == B?
It seems undesirable that equality comparison is permitted, even though it has unexpected behavior.

On Mon, Apr 14, 2008 at 7:57 PM, Nathan Bell <[hidden email]> wrote:
On Mon, Apr 14, 2008 at 4:45 PM, Joseph Turian <[hidden email]> wrote:
> I am having trouble determining equality of sparse matrices.
> Consider this code snippet. Although the sparse matrices appear to be equal,
> z==y returns false (until I convert the matrices to dense matrices).
> What is the problem with the equality test here?

Sparse matrices don't currently support that functionality.  A
workaround could be abs(A-B).nnz == 0

--
Nathan Bell [hidden email]
http://graphics.cs.uiuc.edu/~wnbell/
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user



--
Academic: http://www-etud.iro.umontreal.ca/~turian/
Business: http://www.metaoptimize.com/
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: __eq__ for scipy.sparse not working?

Nathan Bell-4
On Tue, Apr 15, 2008 at 12:37 PM, Joseph Turian <[hidden email]> wrote:
> Is there a reason that sparse matrices don't support this functionality?
> What is actually happening when I try equality testing for A == B?
> It seems undesirable that equality comparison is permitted, even though it
> has unexpected behavior.

I agree that __eq__ should either work or raise an exception.  As to
why __eq__ isn't supported, I haven't written the necessary code to
handle arrays with dtype='bool'.

Offhand, I don't know what specifically needs to be changed to make
sparse matrices agree with numpy's handling of boolean arrays.  Many
of the necessary ingredients are already present, but I have not fully
explored this matter.

I created a ticket in Trac for this issue:
http://scipy.org/scipy/scipy/ticket/639

Unfortunately, time is scarce for me at the moment, so I can't say
when I'll get around to it.

--
Nathan Bell [hidden email]
http://graphics.cs.uiuc.edu/~wnbell/
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: __eq__ for scipy.sparse not working?

Nathan Bell-4
On Tue, Apr 15, 2008 at 2:09 PM, Nathan Bell <[hidden email]> wrote:

> On Tue, Apr 15, 2008 at 12:37 PM, Joseph Turian <[hidden email]> wrote:
>  > Is there a reason that sparse matrices don't support this functionality?
>  > What is actually happening when I try equality testing for A == B?
>  > It seems undesirable that equality comparison is permitted, even though it
>  > has unexpected behavior.
>
>  I agree that __eq__ should either work or raise an exception.  As to
>  why __eq__ isn't supported, I haven't written the necessary code to
>  handle arrays with dtype='bool'.
>

I should also add that some operations cannot be supported in a
straightforward manner.  For instance, (A < 2.0) is not a safe
operation on large sparse matrices.  It's unclear what should be done
in this case.

--
Nathan Bell [hidden email]
http://graphics.cs.uiuc.edu/~wnbell/
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: __eq__ for scipy.sparse not working?

Anne Archibald
In reply to this post by Nathan Bell-4
On 15/04/2008, Nathan Bell <[hidden email]> wrote:

> On Tue, Apr 15, 2008 at 12:37 PM, Joseph Turian <[hidden email]> wrote:
>  > Is there a reason that sparse matrices don't support this functionality?
>  > What is actually happening when I try equality testing for A == B?
>  > It seems undesirable that equality comparison is permitted, even though it
>  > has unexpected behavior.
>
> I agree that __eq__ should either work or raise an exception.  As to
>  why __eq__ isn't supported, I haven't written the necessary code to
>  handle arrays with dtype='bool'.
>
>  Offhand, I don't know what specifically needs to be changed to make
>  sparse matrices agree with numpy's handling of boolean arrays.  Many
>  of the necessary ingredients are already present, but I have not fully
>  explored this matter.
>
>  I created a ticket in Trac for this issue:
>  http://scipy.org/scipy/scipy/ticket/639
>
>  Unfortunately, time is scarce for me at the moment, so I can't say
>  when I'll get around to it.

This is actually tricky: you definitely want "not" to be a reasonable
operation on sparse boolean arrays, which means you can't just store
the "True" values as nonzero entries. It's still doable, with some
sort of flag in the sparse object indicating whether the array as a
whole has been negated, but it's going to be a pain.

Anne
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: __eq__ for scipy.sparse not working?

Joseph Turian
My preference is that if something does not work as expected, then throw an exception. This is better than silently doing the wrong thing, plus it doesn't take much time to code :^)

Best,
  Joseph

On Tue, Apr 15, 2008 at 4:27 PM, Anne Archibald <[hidden email]> wrote:
On 15/04/2008, Nathan Bell <[hidden email]> wrote:
> On Tue, Apr 15, 2008 at 12:37 PM, Joseph Turian <[hidden email]> wrote:
>  > Is there a reason that sparse matrices don't support this functionality?
>  > What is actually happening when I try equality testing for A == B?
>  > It seems undesirable that equality comparison is permitted, even though it
>  > has unexpected behavior.
>
> I agree that __eq__ should either work or raise an exception.  As to
>  why __eq__ isn't supported, I haven't written the necessary code to
>  handle arrays with dtype='bool'.
>
>  Offhand, I don't know what specifically needs to be changed to make
>  sparse matrices agree with numpy's handling of boolean arrays.  Many
>  of the necessary ingredients are already present, but I have not fully
>  explored this matter.
>
>  I created a ticket in Trac for this issue:
>  http://scipy.org/scipy/scipy/ticket/639
>
>  Unfortunately, time is scarce for me at the moment, so I can't say
>  when I'll get around to it.

This is actually tricky: you definitely want "not" to be a reasonable
operation on sparse boolean arrays, which means you can't just store
the "True" values as nonzero entries. It's still doable, with some
sort of flag in the sparse object indicating whether the array as a
whole has been negated, but it's going to be a pain.

Anne
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user



--
Academic: http://www-etud.iro.umontreal.ca/~turian/
Business: http://www.metaoptimize.com/
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user