timeseries - mov_average_expw alters it's input

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

timeseries - mov_average_expw alters it's input

dhirschfeld
As demonstrated below the mov_average_expw function changes its input series.
Is this known or expected behaviour or a bug? I'd venture to suggest it's a
little surprising, especially for new users.

-Dave

 from copy import deepcopy
 import numpy.ma as ma
 from numpy.random import rand
 import scikits.timeseries as ts
 from scikits.timeseries.lib.moving_funcs import mov_average_expw

 N = 256
 series = ts.time_series(rand(N),
                 ts.date_array(start_date=ts.Date('D','2008-01-01'),length=N))
 series[96:128] = ma.masked
 original_series = deepcopy(series)
 filtered_series = mov_average_expw(series,16)

 assert (series.mask == original_series.mask).all()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

 assert (filtered_series.mask == series.mask).all()

 ts.__version__
'0.91.1'
 import numpy as np; np.__version__
'1.4.0.dev6882'

_______________________________________________
SciPy-user mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: timeseries - mov_average_expw alters it's input

josef.pktd
On Fri, May 8, 2009 at 9:21 AM, Dave Hirschfeld
<[hidden email]> wrote:

> As demonstrated below the mov_average_expw function changes its input series.
> Is this known or expected behaviour or a bug? I'd venture to suggest it's a
> little surprising, especially for new users.
>
> -Dave
>
>  from copy import deepcopy
>  import numpy.ma as ma
>  from numpy.random import rand
>  import scikits.timeseries as ts
>  from scikits.timeseries.lib.moving_funcs import mov_average_expw
>
>  N = 256
>  series = ts.time_series(rand(N),
>                 ts.date_array(start_date=ts.Date('D','2008-01-01'),length=N))
>  series[96:128] = ma.masked
>  original_series = deepcopy(series)
>  filtered_series = mov_average_expw(series,16)
>
>  assert (series.mask == original_series.mask).all()
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AssertionError
>
>  assert (filtered_series.mask == series.mask).all()
>
>  ts.__version__
> '0.91.1'
>  import numpy as np; np.__version__
> '1.4.0.dev6882'
>

I wouldn't be surprised. What would be the moving average of your
observations starting at 129, when the previous observations are
masked?

Maybe you can try to change the ``tol`` parameter, to get the result you want?


tol : {1e-6, float}, optional

    Tolerance for the definition of the mask. When data contains
masked values, this parameter determines what points in the result
should be masked. Values in the result that would not be
“significantly” impacted (as determined by this parameter) by the
masked values are left unmasked.

Josef
_______________________________________________
SciPy-user mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: timeseries - mov_average_expw alters it's input

Pierre GM-2
Dave,
It looks like a bug indeed, the mask of the original series shouldn't  
be modified. I'm on it and will let you know when it's fixed.
(BTW, I advise you to use numpy.ma.testutils.assert_equal to test the  
equality of 2 MaskedArrays, instead of the syntax you were using in  
the example)

On May 8, 2009, at 10:30 AM, [hidden email] wrote:

> On Fri, May 8, 2009 at 9:21 AM, Dave Hirschfeld
> <[hidden email]> wrote:
>> As demonstrated below the mov_average_expw function changes its  
>> input series.
>> Is this known or expected behaviour or a bug? I'd venture to  
>> suggest it's a
>> little surprising, especially for new users.
>>
>> -Dave
>>
>>  from copy import deepcopy
>>  import numpy.ma as ma
>>  from numpy.random import rand
>>  import scikits.timeseries as ts
>>  from scikits.timeseries.lib.moving_funcs import mov_average_expw
>>
>>  N = 256
>>  series = ts.time_series(rand(N),
>>                  
>> ts.date_array(start_date=ts.Date('D','2008-01-01'),length=N))
>>  series[96:128] = ma.masked
>>  original_series = deepcopy(series)
>>  filtered_series = mov_average_expw(series,16)
>>
>>  assert (series.mask == original_series.mask).all()
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>> AssertionError
>>
>>  assert (filtered_series.mask == series.mask).all()
>>
>>  ts.__version__
>> '0.91.1'
>>  import numpy as np; np.__version__
>> '1.4.0.dev6882'
>>
>
> I wouldn't be surprised. What would be the moving average of your
> observations starting at 129, when the previous observations are
> masked?
>
> Maybe you can try to change the ``tol`` parameter, to get the result  
> you want?
>
>
> tol : {1e-6, float}, optional
>
>    Tolerance for the definition of the mask. When data contains
> masked values, this parameter determines what points in the result
> should be masked. Values in the result that would not be
> “significantly” impacted (as determined by this parameter) by the
> masked values are left unmasked.
>
> Josef
> _______________________________________________
> SciPy-user mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-user mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: timeseries - mov_average_expw alters it's input

Pierre GM-2
In reply to this post by dhirschfeld
Dave,
That should be fixed in the SVN (r2187). Do you want to give it a try ?
Thanks a lot again for reporting.
P.

On May 8, 2009, at 9:21 AM, Dave Hirschfeld wrote:

> As demonstrated below the mov_average_expw function changes its  
> input series.
> Is this known or expected behaviour or a bug? I'd venture to suggest  
> it's a
> little surprising, especially for new users.
>
> -Dave
>
> from copy import deepcopy
> import numpy.ma as ma
> from numpy.random import rand
> import scikits.timeseries as ts
> from scikits.timeseries.lib.moving_funcs import mov_average_expw
>
> N = 256
> series = ts.time_series(rand(N),
>                  
> ts.date_array(start_date=ts.Date('D','2008-01-01'),length=N))
> series[96:128] = ma.masked
> original_series = deepcopy(series)
> filtered_series = mov_average_expw(series,16)
>
> assert (series.mask == original_series.mask).all()
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AssertionError
>
> assert (filtered_series.mask == series.mask).all()
>
> ts.__version__
> '0.91.1'
> import numpy as np; np.__version__
> '1.4.0.dev6882'
>
> _______________________________________________
> SciPy-user mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-user mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: timeseries - mov_average_expw alters it's input

dhirschfeld
Pierre GM <pgmdevlist <at> gmail.com> writes:

>
> Dave,
> That should be fixed in the SVN (r2187). Do you want to give it a try ?
> Thanks a lot again for reporting.
> P.
>
>(BTW, I advise you to use numpy.ma.testutils.assert_equal to test the  
>equality of 2 MaskedArrays, instead of the syntax you were using in  
>the example)
>

Thanks for the fix (and the info!) That has indeed solved the problem.

-Dave

_______________________________________________
SciPy-user mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user