iteratively masking timeseries

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

iteratively masking timeseries

Timmie
Administrator
Hello,
I am making my way forward into timeseries processing with the scikit
package.

I have a timeseries where NoData values are masked when loading the data
into the timeseries.

Now I would like to apply some filters on the data like discarding data
values below measurement device accuracy or above a certain threshold.

Therefore I followed the appraoch outlined in the FAQ [1].
But then I get the pasted below at the end.

Does that mean that one can only mask an array once and cannot mask more
values later on?

I would like to do something like:

1) create timeseries with NoData values => already can do that
2) apply various filters masking more and more data.
        a) e.g. get a series with masked values below 10.
        b) e.g. get a series with masked values above 100.

I would appreciate any help or hint here.

Kind regards,
Tim


#### pasted from Ipython ####

In [18]: mask[mask<0] = numpy.ma.masked

---------------------------------------------------------------------------

IndexError                                Traceback (most recent call last)



D:\scripts\timeseries.py i

----> 1

       2

       3

       4

       5



C:\python25\lib\site-packages\scikits\timeseries\tseries.pyc in
__setitem__(se

     522         if self is masked:

     523             raise MAError, 'Cannot alter the masked element.'

--> 524         (sindx, _) = self.__checkindex(indx)

     525         super(TimeSeries, self).__setitem__(sindx, value)

     526     #......................................................



C:\python25\lib\site-packages\scikits\timeseries\tseries.pyc in
__checkindex(s

     490             msg = "Masked arrays must be filled before they can
be use

     491                   "as indices!"

--> 492             raise IndexError, msg

     493         return (indx,indx)

     494



IndexError: Masked arrays must be filled before they can be used as indices!

#### end from Ipython ####

[1]
http://www.scipy.org/Cookbook/TimeSeries/FAQ#head-cfe3617dda0b030f0474a2a773e2dca4da8eaea0

_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: iteratively masking timeseries

Pierre GM-2
On Thursday 12 June 2008 17:14:03 Tim Michelsen wrote:
> Does that mean that one can only mask an array once and cannot mask more
> values later on?

No no, the message
> IndexError: Masked arrays must be filled before they can be used as
> indices!

doesn't mean you can't mask arrays more than once, just that you can't use a
MaskedArray as index in a TimeSeries without having to fill it first. The
reason for this behavior is that we can't tell beforehand how you want to
deal with your masked values: should a masked value be considered True ?
False ?

So, when you want to do something like:
> mask[mask<0] = numpy.ma.masked

just do:

mask[(mask<0).filled(True)] = numpy.ma.masked

That way, you're masking the values of mask that are negative, and keep the
masked values as masked.

_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user