[CC'ing scipy-dev because see below]
On Tue, Sep 24, 2013 at 6:45 PM, Nathaniel Smith <
[hidden email]> wrote:
> Hi all,
>
> I'm getting a very strange error, and unfortunately I can't seem to
> reproduce it *except* when running on Travis-CI, so my debugging tools
> are really limited. I'm wondering if anyone else has seen anything
> like this?
>
> The offending line of code is:
>
> File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/pyrerp/incremental_ls.py",
> line 323, in append_bottom_half
> self.xtx += xtx
> ValueError: setting an array element with a sequence.
>
> And debug prints reveal that in the case that causes the error,
> 'self.xtx' is an ndarray that prints as:
>
> [[ 0. 0.]
> [ 0. 0.]]
>
> while 'xtx' is a scipy.sparse.csc.csc_matrix that prints as:
>
> (1, 0) 45.0
> (0, 0) 10.0
> (1, 1) 285.0
> (0, 1) 45.0
In accordance with the cosmic law governing such things, this turns
out to be a bug in numpy that I introduced in 2397c9d4, specifically
this line, which introduces an unchecked DEPRECATE:
https://github.com/numpy/numpy/commit/2397c9d4#L5R528(Plus some complicated nonsense involving virtualenvs that sometimes
fall back on system libraries even though the virtualenv contains a
perfectly good version of the same library etc. to make reproduction
more confusing.)
My test code was setting warnings to raise errors by default, and
apparently ndarray += csc_matrix goes through some circuitous path
that (AFAICT) involves creating an object ndarray containing the
csc_matrix and calling the add ufunc, which somehow trips on the cast
warning above.
Then later on, at line 159 of arraytypes.c.src, the the @TYPE@_setitem
function does:
if (PyErr_Occurred()) {
if (PySequence_Check(op) && !PyString_Check(op) &&
!PyUnicode_Check(op)) {
PyErr_Clear();
PyErr_SetString(PyExc_ValueError,
"setting an array element with a sequence.");
}
return -1;
}
and the PyErr_Occurred here catches the real error and replaces it
with some nonsense.
So three points:
1) Maybe this will be useful to someone googling later.
2) I guess I'll file some horrible patch against numpy that just
throws away exceptions caused by that deprecation, because there is no
way for PyArray_CanCastTypeTo to raise an error :-(.
3) This script raises a DeprecationWarning with numpy 1.7.1 and scipy 0.12.0:
import warnings, numpy, scipy.sparse
warnings.filterwarnings("always")
a = numpy.zeros((2, 2))
b = scipy.sparse.csc_matrix([[0.0, 0.0], [0.0, 0.0]])
a += b
test-script3.py:5: DeprecationWarning: Implicitly casting between
incompatible kinds. In a future numpy release, this will raise an
error. Use casting="unsafe" if this is intentional.
I really don't understand what arcane magic is used to make ndarray +=
csc_matrix work at all, but my question is, is it going to break when
we complete the casting transition described above? It was just
supposed to catch things like int += float.
-n
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Locked
