[SciPy-User] Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

Ergun Bicici-2

After multiplying a coo_matrix from scipy.sparse with its copy, the copy's format changes to column-major and hence its data change while the original data remains the same.

Example:
import numpy, scipy
from scipy.sparse import coo_matrix, csc_matrix

a = numpy.random.randn(4,7)
print ('a:', a)
b = coo_matrix(a)
c = b.copy()
print ('c before (coo_matrix):', c)
print ('c data before (coo_matrix):', c.data)
b.multiply(c)
print ('c after (coo_matrix):', c)
print ('c data after (coo_matrix):', c.data)

a: [[ 0.80168251 -1.39944231  0.25345622  1.42371329 -0.12530735  0.50449369
  1.07289013]
[ 0.45875588 -0.15916679 -0.42632756  0.56484416 -1.46640872  1.42488646
  0.03532664]
[ 0.72752298  1.34140445 -0.37043453  0.78127267  0.01084105 -2.08957963
 -0.10899932]
[-0.02555597 -0.87197938 -0.76835675  0.82776032 -2.06292275 -1.2911363
 -0.72253431]]
c before (coo_matrix):   (0, 0) 0.801682509603
 (0, 1)        -1.39944230956
 (0, 2)        0.253456223565
 (0, 3)        1.4237132942
 (0, 4)        -0.125307347426
 (0, 5)        0.504493692248
 (0, 6)        1.07289013113
 (1, 0)        0.458755875333
 (1, 1)        -0.159166794146
 (1, 2)        -0.426327564398
 (1, 3)        0.564844163879
 (1, 4)        -1.46640871661
 (1, 5)        1.42488646441
 (1, 6)        0.0353266410166
 (2, 0)        0.727522979291
 (2, 1)        1.34140444948
 (2, 2)        -0.370434529077
 (2, 3)        0.781272667369
 (2, 4)        0.0108410499861
 (2, 5)        -2.08957962801
 (2, 6)        -0.108999316299
 (3, 0)        -0.0255559745129
 (3, 1)        -0.871979375733
 (3, 2)        -0.768356754242
 (3, 3)        0.827760318972
 (3, 4)        -2.0629227477
 (3, 5)        -1.29113629796
 (3, 6)        -0.722534309773
c data before (coo_matrix): [ 0.80168251 -1.39944231  0.25345622  1.42371329 -0.12530735  0.50449369
 1.07289013  0.45875588 -0.15916679 -0.42632756  0.56484416 -1.46640872
 1.42488646  0.03532664  0.72752298  1.34140445 -0.37043453  0.78127267
 0.01084105 -2.08957963 -0.10899932 -0.02555597 -0.87197938 -0.76835675
 0.82776032 -2.06292275 -1.2911363  -0.72253431]
c after (coo_matrix):   (0, 0)  0.801682509603
 (1, 0)        0.458755875333
 (2, 0)        0.727522979291
 (3, 0)        -0.0255559745129
 (0, 1)        -1.39944230956
 (1, 1)        -0.159166794146
 (2, 1)        1.34140444948
 (3, 1)        -0.871979375733
 (0, 2)        0.253456223565
 (1, 2)        -0.426327564398
 (2, 2)        -0.370434529077
 (3, 2)        -0.768356754242
 (0, 3)        1.4237132942
 (1, 3)        0.564844163879
 (2, 3)        0.781272667369
 (3, 3)        0.827760318972
 (0, 4)        -0.125307347426
 (1, 4)        -1.46640871661
 (2, 4)        0.0108410499861
 (3, 4)        -2.0629227477
 (0, 5)        0.504493692248
 (1, 5)        1.42488646441
 (2, 5)        -2.08957962801
 (3, 5)        -1.29113629796
 (0, 6)        1.07289013113
 (1, 6)        0.0353266410166
 (2, 6)        -0.108999316299
 (3, 6)        -0.722534309773
c data after (coo_matrix): [ 0.80168251  0.45875588  0.72752298 -0.02555597 -1.39944231 -0.15916679
 1.34140445 -0.87197938  0.25345622 -0.42632756 -0.37043453 -0.76835675
 1.42371329  0.56484416  0.78127267  0.82776032 -0.12530735 -1.46640872
 0.01084105 -2.06292275  0.50449369  1.42488646 -2.08957963 -1.2911363
 1.07289013  0.03532664 -0.10899932 -0.72253431]


Best Regards,
Ergun

Ergun Biçici


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

David Hagen
What version are you using? I am on 1.0.0 on Windows 7 and can't reproduce this.


In [1]: import numpy, scipy
   ...: from scipy.sparse import coo_matrix, csc_matrix
   ...: 
   ...: a = numpy.random.randn(4,7)
   ...: print ('a:', a)
   ...: b = coo_matrix(a)
   ...: c = b.copy()
   ...: print ('c before (coo_matrix):', c)
   ...: print ('c data before (coo_matrix):', c.data)
   ...: b.multiply(c)
   ...: print ('c after (coo_matrix):', c)
   ...: print ('c data after (coo_matrix):', c.data)

a: [[ 0.13836937 -0.44769877 -0.67486584 -0.18079949  0.68758066 -0.77756412   0.77819897]
 [-0.07750618  0.45723152  0.45801638  0.69517378  0.03544761  0.09793786   0.13236508]
 [-0.65393023  1.1196121   0.45519321  0.27690164 -1.28679679 -0.46791624   0.59063279]
 [ 1.43179076  0.38644705 -0.27769729  1.09228815 -1.38605657 -1.39493734  -1.06365405]]
c before (coo_matrix):   (0, 0) 0.13836937102833774
  (0, 1)        -0.44769876721981233
  (0, 2)        -0.6748658374897415
  (0, 3)        -0.18079948538318338
  (0, 4)        0.6875806622877358
  (0, 5)        -0.7775641155122853
  (0, 6)        0.7781989700714793
  (1, 0)        -0.07750617928808383
  (1, 1)        0.4572315207281202
  (1, 2)        0.4580163825027164
  (1, 3)        0.6951737836620446
  (1, 4)        0.035447606524930446
  (1, 5)        0.09793785855383845
  (1, 6)        0.13236508271088915
  (2, 0)        -0.6539302277704122
  (2, 1)        1.1196120969391556
  (2, 2)        0.45519320523897316
  (2, 3)        0.27690163859596667
  (2, 4)        -1.2867967876681026
  (2, 5)        -0.4679162379777137
  (2, 6)        0.5906327892567922
  (3, 0)        1.4317907565386616
  (3, 1)        0.38644704937601526
  (3, 2)        -0.2776972862955177
  (3, 3)        1.0922881499931638
  (3, 4)        -1.3860565664680384
  (3, 5)        -1.3949373395890408
  (3, 6)        -1.0636540485066541
c data before (coo_matrix): [ 0.13836937 -0.44769877 -0.67486584 -0.18079949  0.68758066 -0.77756412
  0.77819897 -0.07750618  0.45723152  0.45801638  0.69517378  0.03544761
  0.09793786  0.13236508 -0.65393023  1.1196121   0.45519321  0.27690164
 -1.28679679 -0.46791624  0.59063279  1.43179076  0.38644705 -0.27769729
  1.09228815 -1.38605657 -1.39493734 -1.06365405]
c after (coo_matrix):   (0, 0)  0.13836937102833774
  (0, 1)        -0.44769876721981233
  (0, 2)        -0.6748658374897415
  (0, 3)        -0.18079948538318338
  (0, 4)        0.6875806622877358
  (0, 5)        -0.7775641155122853
  (0, 6)        0.7781989700714793
  (1, 0)        -0.07750617928808383
  (1, 1)        0.4572315207281202
  (1, 2)        0.4580163825027164
  (1, 3)        0.6951737836620446
  (1, 4)        0.035447606524930446
  (1, 5)        0.09793785855383845
  (1, 6)        0.13236508271088915
  (2, 0)        -0.6539302277704122
  (2, 1)        1.1196120969391556
  (2, 2)        0.45519320523897316
  (2, 3)        0.27690163859596667
  (2, 4)        -1.2867967876681026
  (2, 5)        -0.4679162379777137
  (2, 6)        0.5906327892567922
  (3, 0)        1.4317907565386616
  (3, 1)        0.38644704937601526
  (3, 2)        -0.2776972862955177
  (3, 3)        1.0922881499931638
  (3, 4)        -1.3860565664680384
  (3, 5)        -1.3949373395890408
  (3, 6)        -1.0636540485066541
c data after (coo_matrix): [ 0.13836937 -0.44769877 -0.67486584 -0.18079949  0.68758066 -0.77756412
  0.77819897 -0.07750618  0.45723152  0.45801638  0.69517378  0.03544761
  0.09793786  0.13236508 -0.65393023  1.1196121   0.45519321  0.27690164
 -1.28679679 -0.46791624  0.59063279  1.43179076  0.38644705 -0.27769729
  1.09228815 -1.38605657 -1.39493734 -1.06365405]

In [2]: scipy.version.version
Out[2]: '1.0.0'

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

Ergun Bicici-2

Thank you. The versions I use on linux are

scipy.__version__
'0.19.1'

numpy.__version__
'1.13.3'

​Can you try on linux as well?​


Best Regards,
Ergun

Ergun Biçici


On Wed, Mar 21, 2018 at 1:48 PM, David Hagen <[hidden email]> wrote:
What version are you using? I am on 1.0.0 on Windows 7 and can't reproduce this.


In [1]: import numpy, scipy
   ...: from scipy.sparse import coo_matrix, csc_matrix
   ...: 
   ...: a = numpy.random.randn(4,7)
   ...: print ('a:', a)
   ...: b = coo_matrix(a)
   ...: c = b.copy()
   ...: print ('c before (coo_matrix):', c)
   ...: print ('c data before (coo_matrix):', c.data)
   ...: b.multiply(c)
   ...: print ('c after (coo_matrix):', c)
   ...: print ('c data after (coo_matrix):', c.data)

a: [[ 0.13836937 -0.44769877 -0.67486584 -0.18079949  0.68758066 -0.77756412   0.77819897]
 [-0.07750618  0.45723152  0.45801638  0.69517378  0.03544761  0.09793786   0.13236508]
 [-0.65393023  1.1196121   0.45519321  0.27690164 -1.28679679 -0.46791624   0.59063279]
 [ 1.43179076  0.38644705 -0.27769729  1.09228815 -1.38605657 -1.39493734  -1.06365405]]
c before (coo_matrix):   (0, 0) 0.13836937102833774
  (0, 1)        -0.44769876721981233
  (0, 2)        -0.6748658374897415
  (0, 3)        -0.18079948538318338
  (0, 4)        0.6875806622877358
  (0, 5)        -0.7775641155122853
  (0, 6)        0.7781989700714793
  (1, 0)        -0.07750617928808383
  (1, 1)        0.4572315207281202
  (1, 2)        0.4580163825027164
  (1, 3)        0.6951737836620446
  (1, 4)        0.035447606524930446
  (1, 5)        0.09793785855383845
  (1, 6)        0.13236508271088915
  (2, 0)        -0.6539302277704122
  (2, 1)        1.1196120969391556
  (2, 2)        0.45519320523897316
  (2, 3)        0.27690163859596667
  (2, 4)        -1.2867967876681026
  (2, 5)        -0.4679162379777137
  (2, 6)        0.5906327892567922
  (3, 0)        1.4317907565386616
  (3, 1)        0.38644704937601526
  (3, 2)        -0.2776972862955177
  (3, 3)        1.0922881499931638
  (3, 4)        -1.3860565664680384
  (3, 5)        -1.3949373395890408
  (3, 6)        -1.0636540485066541
c data before (coo_matrix): [ 0.13836937 -0.44769877 -0.67486584 -0.18079949  0.68758066 -0.77756412
  0.77819897 -0.07750618  0.45723152  0.45801638  0.69517378  0.03544761
  0.09793786  0.13236508 -0.65393023  1.1196121   0.45519321  0.27690164
 -1.28679679 -0.46791624  0.59063279  1.43179076  0.38644705 -0.27769729
  1.09228815 -1.38605657 -1.39493734 -1.06365405]
c after (coo_matrix):   (0, 0)  0.13836937102833774
  (0, 1)        -0.44769876721981233
  (0, 2)        -0.6748658374897415
  (0, 3)        -0.18079948538318338
  (0, 4)        0.6875806622877358
  (0, 5)        -0.7775641155122853
  (0, 6)        0.7781989700714793
  (1, 0)        -0.07750617928808383
  (1, 1)        0.4572315207281202
  (1, 2)        0.4580163825027164
  (1, 3)        0.6951737836620446
  (1, 4)        0.035447606524930446
  (1, 5)        0.09793785855383845
  (1, 6)        0.13236508271088915
  (2, 0)        -0.6539302277704122
  (2, 1)        1.1196120969391556
  (2, 2)        0.45519320523897316
  (2, 3)        0.27690163859596667
  (2, 4)        -1.2867967876681026
  (2, 5)        -0.4679162379777137
  (2, 6)        0.5906327892567922
  (3, 0)        1.4317907565386616
  (3, 1)        0.38644704937601526
  (3, 2)        -0.2776972862955177
  (3, 3)        1.0922881499931638
  (3, 4)        -1.3860565664680384
  (3, 5)        -1.3949373395890408
  (3, 6)        -1.0636540485066541
c data after (coo_matrix): [ 0.13836937 -0.44769877 -0.67486584 -0.18079949  0.68758066 -0.77756412
  0.77819897 -0.07750618  0.45723152  0.45801638  0.69517378  0.03544761
  0.09793786  0.13236508 -0.65393023  1.1196121   0.45519321  0.27690164
 -1.28679679 -0.46791624  0.59063279  1.43179076  0.38644705 -0.27769729
  1.09228815 -1.38605657 -1.39493734 -1.06365405]

In [2]: scipy.version.version
Out[2]: '1.0.0'

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

David Hagen
​Can you try on linux as well?​

Sorry, I do not have access to a linux box right now. Can you try 1.0.0 in virtual environment?

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

Ergun Bicici-2

I'll wait for response from others. Your environment looks different. Thank you.


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 3:16 PM, David Hagen <[hidden email]> wrote:
​Can you try on linux as well?​

Sorry, I do not have access to a linux box right now. Can you try 1.0.0 in virtual environment?

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

Juan Nunez-Iglesias
Hi Ergun,

You should probably update your SciPy version to 1.0. It is more likely that the “fixed” behaviour reported by David is indeed fixed in 1.0, rather than an OS difference.

Juan.

On 22 Mar 2018, 10:59 AM -0400, Ergun Bicici <[hidden email]>, wrote:

I'll wait for response from others. Your environment looks different. Thank you.


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 3:16 PM, David Hagen <[hidden email]> wrote:
​Can you try on linux as well?​

Sorry, I do not have access to a linux box right now. Can you try 1.0.0 in virtual environment?

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

Ergun Bicici-2

I installed newer scipy using pip3 install scipy and there is no error this time. 

The test case might still be useful to test sparse matrices and their data.


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 11:41 PM, Juan Nunez-Iglesias <[hidden email]> wrote:
Hi Ergun,

You should probably update your SciPy version to 1.0. It is more likely that the “fixed” behaviour reported by David is indeed fixed in 1.0, rather than an OS difference.

Juan.

On 22 Mar 2018, 10:59 AM -0400, Ergun Bicici <[hidden email]>, wrote:

I'll wait for response from others. Your environment looks different. Thank you.


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 3:16 PM, David Hagen <[hidden email]> wrote:
​Can you try on linux as well?​

Sorry, I do not have access to a linux box right now. Can you try 1.0.0 in virtual environment?

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

Juan Nunez-Iglesias
In reply to this post by Juan Nunez-Iglesias
Actually, I hadn’t, and I realised I hadn’t updated scipy to 1.0, so it was a good chance to try both. I’m on macOS. Indeed, on 0.19 I see the same issue you do, while on 1.0 it is gone.

Incidentally, there’s a few issues related to this that might have been resolved without closing the corresponding GH issue:

These may have been fixed here:

Any devs want to comment on these?

Juan.

On 22 Mar 2018, 6:22 PM -0400, Ergun Bicici <[hidden email]>, wrote:

Hi Juan,

Ok I am installing:
pip3 install scipy
How about you? Did you try?


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 11:41 PM, Juan Nunez-Iglesias < [hidden email]> wrote:
Hi Ergun,

You should probably update your SciPy version to 1.0. It is more likely that the “fixed” behaviour reported by David is indeed fixed in 1.0, rather than an OS difference.

Juan.

On 22 Mar 2018, 10:59 AM -0400, Ergun Bicici <[hidden email]>, wrote:

I'll wait for response from others. Your environment looks different. Thank you.


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 3:16 PM, David Hagen <[hidden email]> wrote:
​Can you try on linux as well?​

Sorry, I do not have access to a linux box right now. Can you try 1.0.0 in virtual environment?

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Error in scipy sparse: multiplication with coo_matrix copy changes the data and the format of the multiplied

Ergun Bicici-2

Hi Juan,

Thank you for your confirmation. The issues you mentioned are also different. 


Best Regards,
Ergun

Ergun Biçici


On Fri, Mar 23, 2018 at 2:00 AM, Juan Nunez-Iglesias <[hidden email]> wrote:
Actually, I hadn’t, and I realised I hadn’t updated scipy to 1.0, so it was a good chance to try both. I’m on macOS. Indeed, on 0.19 I see the same issue you do, while on 1.0 it is gone.

Incidentally, there’s a few issues related to this that might have been resolved without closing the corresponding GH issue:

These may have been fixed here:

Any devs want to comment on these?

Juan.

On 22 Mar 2018, 6:22 PM -0400, Ergun Bicici <[hidden email]>, wrote:

Hi Juan,

Ok I am installing:
pip3 install scipy
How about you? Did you try?


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 11:41 PM, Juan Nunez-Iglesias < [hidden email]> wrote:
Hi Ergun,

You should probably update your SciPy version to 1.0. It is more likely that the “fixed” behaviour reported by David is indeed fixed in 1.0, rather than an OS difference.

Juan.

On 22 Mar 2018, 10:59 AM -0400, Ergun Bicici <[hidden email]>, wrote:

I'll wait for response from others. Your environment looks different. Thank you.


Best Regards,
Ergun

Ergun Biçici


On Thu, Mar 22, 2018 at 3:16 PM, David Hagen <[hidden email]> wrote:
​Can you try on linux as well?​

Sorry, I do not have access to a linux box right now. Can you try 1.0.0 in virtual environment?

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user


_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user