scikits.timeseries : how would I plot (or calculate) monthly statistics.

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

scikits.timeseries : how would I plot (or calculate) monthly statistics.

Dharhas Pothina
Hi,

This a question for those of you who use scikits.timeseries .

Given 10-15 years of timeseries data how would I plot monthly statistics like max, min, mean, std deviation etc for a year.

ie. on the x axis would be the 12 months (Jan,Feb...Dec) and y axis would have the stats.

I know how to read in my timeseries and convert it to a monthly frequency but I'm not sure the best way to proceed from here. I guess one way would be to reshape the monthly timeseries array into 12 columns and then calculate the stats on each column but I was wondering if there was a more appropriate method to do this?

On another note: shouldn't there be a link on this page (http://scikits.appspot.com/timeseries) to the timeseries home page http://pytseries.sourceforge.net/  . The link on the appspot page (http://pypi.python.org/pypi/scikits.timeseries) doesn't work.

thanks

- dharhas

_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries : how would I plot (or calculate) monthly statistics.

Timmie
Administrator
Hello

> Given 10-15 years of timeseries data how would I plot monthly
> statistics like max, min, mean, std deviation etc for a year.
Something like:

import scikits.timeseries as ts
import numpy as np
start_data = ts.Date(freq='M', year=1990, month=1)
data = np.random.uniform(0, 20, 120)
ts_monthly = ts.time_series(data, freq='M', start_date=start_data)
aser = ts_monthly.convert('A', func=np.ma.std)

=> now plot aser?

Regards,
Timmie

P.S.: regarding time series, I have sth. for you . Please pass me your PM.

_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries : how would I plot (or calculate) monthly statistics.

Pierre GM-2
In reply to this post by Dharhas Pothina

On Mar 4, 2009, at 11:15 AM, Dharhas Pothina wrote:
>
> Given 10-15 years of timeseries data

What frequency ? Monthly ? Daily ?

>
> I know how to read in my timeseries and convert it to a monthly  
> frequency but I'm not sure the best way to proceed from here. I  
> guess one way would be to reshape the monthly timeseries array into  
> 12 columns and then calculate the stats on each column but I was  
> wondering if there was a more appropriate method to do this?

* If you have a monhtly frequency: just use .convert('A') and compute  
the stats for each column.

* If you have a daily frequency:
well, I'm afraid you didn't give enough information: what do you want  
for each month, what is the expected shape of the output ? Statistics  
far all the years (eg, the mean for January irrespectively of the  
year), with an output of shape (12,) ? For each year, with an output  
of shape (n, 12) and n the nb of years?
The generic idea is indeed to first convert to monthly. You end up  
with a (12*n, 31) series that you have to reshape. However, because  
you won't need the dates, I strongly advise you to work only on  
the .series part for the reshaping.


> On another note: shouldn't there be a link on this page (http://scikits.appspot.com/timeseries 
> ) to the timeseries home page http://pytseries.sourceforge.net/  .  
> The link on the appspot page (http://pypi.python.org/pypi/scikits.timeseries 
> ) doesn't work.

Thanks for reporting. We were already aware of the problem, that shall  
be fixed when timeseries will be officially released (and then, we'll  
be able to put it on pypi).
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries : how would I plot (orcalculate) monthly statistics.

Dharhas Pothina

Original data is on an approximately two week frequency. I'll be reading them in as daily values and converting to monthly.

I eventually want to calculate plot monthly values in a particular year vs mean monthly values for all years (ie vs mean for each month irrespective of year)

I'll try playing around with the what you and Tim have mentioned. but I'm a bit confused about how some of this works.

> If you have a monthly frequency: just use .convert('A') and compute  
> the stats for each column.

Wouldn't this convert it to an Annual frequency, ie one value per year?

> The generic idea is indeed to first convert to monthly. You end up  
> with a (12*n, 31) series that you have to reshape

I get the 12*n , ie 12 months * number of years. Where does the 31 come from. Does the series still contain the daily values after conversion to monthly?

thanks,

- dharhas

>>> Pierre GM <[hidden email]> 3/4/2009 2:38 PM >>>

On Mar 4, 2009, at 11:15 AM, Dharhas Pothina wrote:
>
> Given 10-15 years of timeseries data

What frequency ? Monthly ? Daily ?

>
> I know how to read in my timeseries and convert it to a monthly  
> frequency but I'm not sure the best way to proceed from here. I  
> guess one way would be to reshape the monthly timeseries array into  
> 12 columns and then calculate the stats on each column but I was  
> wondering if there was a more appropriate method to do this?

* If you have a monhtly frequency: just use .convert('A') and compute  
the stats for each column.

* If you have a daily frequency:
well, I'm afraid you didn't give enough information: what do you want  
for each month, what is the expected shape of the output ? Statistics  
far all the years (eg, the mean for January irrespectively of the  
year), with an output of shape (12,) ? For each year, with an output  
of shape (n, 12) and n the nb of years?
The generic idea is indeed to first convert to monthly. You end up  
with a (12*n, 31) series that you have to reshape. However, because  
you won't need the dates, I strongly advise you to work only on  
the .series part for the reshaping.


> On another note: shouldn't there be a link on this page (http://scikits.appspot.com/timeseries 
> ) to the timeseries home page http://pytseries.sourceforge.net/  .  
> The link on the appspot page (http://pypi.python.org/pypi/scikits.timeseries 
> ) doesn't work.

Thanks for reporting. We were already aware of the problem, that shall  
be fixed when timeseries will be officially released (and then, we'll  
be able to put it on pypi).
_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries : how would I plot (orcalculate)monthly statistics.

Timmie
Administrator
> Original data is on an approximately two week frequency. I'll be reading them
in as daily values and
> converting to monthly.
It would be helpful if you could create an example maybe using random data.

> > If you have a monthly frequency: just use .convert('A') and compute  
> > the stats for each column.
>
> Wouldn't this convert it to an Annual frequency, ie one value per year?
Yes, I though this is what you wanted.

If not maybe you want to take a look at:
scikits.timeseries.extrax.convert_to_annual(series)

in
http://bazaar.launchpad.net/~vcs-imports/scipy/timeseries/annotate/head%3A/scikits/timeseries//extras.py


This will align the years. Then you can create an average over the years.

Greetings,
Timmie

_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries : how would I plot (orcalculate) monthly statistics.

Pierre GM-2
In reply to this post by Dharhas Pothina
Dharhas,

On Mar 4, 2009, at 4:26 PM, Dharhas Pothina wrote:
> Original data is on an approximately two week frequency. I'll be  
> reading them in as daily values and converting to monthly.

OK, you could try 'W', but 'D' is also OK.

> I eventually want to calculate plot monthly values in a particular  
> year vs mean monthly values for all years (ie vs mean for each month  
> irrespective of year)

Then, that's quite easy:
First, compute the means per month:
 >>> mseries = series.convert('M', func=ma.mean)
You end up with a (n*12) series. Convert it to annual, but w/o  
precising a function
 >>> aseries = mseries.convert('A')
aseries is a (n, 12) series: years in rows, months in columns.  
aseries[i,j] is the mean over month j for year i.

If you want the monthly values over all the years, just take the mean  
along axis=0

>
>
>> If you have a monthly frequency: just use .convert('A') and compute
>> the stats for each column.
>
> Wouldn't this convert it to an Annual frequency, ie one value per  
> year?

It does, but because you don't precise a func, you end up with a 2D  
array

http://pytseries.sourceforge.net/generated/scikits.timeseries.TimeSeries.convert.html#scikits.timeseries.TimeSeries.convert


>> The generic idea is indeed to first convert to monthly. You end up
>> with a (12*n, 31) series that you have to reshape
>
> I get the 12*n , ie 12 months * number of years. Where does the 31  
> come from. Does the series still contain the daily values after  
> conversion to monthly?

I was talking about a 'D'aily series to convert to monthly, without  
using a 'func' parameter. You then end up with a 2D series, with 12*n  
rows and 31 columns (because there's at most 31 days per month). Note  
that the last column (day 31) would be masked for Feb, Apr, Jun, Sep,  
Nov.


_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries : how would Iplot (orcalculate) monthly statistics.

Dharhas Pothina

Thank you. This worked great and saved me a lot of time. I'm slowly getting the hang of this package. Sorry for the obvious question but I assume that ma.mean()  etc automatically excludes masked values from the calculation.

- d

>>> Pierre GM <[hidden email]> 3/4/2009 3:46 PM >>>
Dharhas,

On Mar 4, 2009, at 4:26 PM, Dharhas Pothina wrote:
> Original data is on an approximately two week frequency. I'll be  
> reading them in as daily values and converting to monthly.

OK, you could try 'W', but 'D' is also OK.

> I eventually want to calculate plot monthly values in a particular  
> year vs mean monthly values for all years (ie vs mean for each month  
> irrespective of year)

Then, that's quite easy:
First, compute the means per month:
 >>> mseries = series.convert('M', func=ma.mean)
You end up with a (n*12) series. Convert it to annual, but w/o  
precising a function
 >>> aseries = mseries.convert('A')
aseries is a (n, 12) series: years in rows, months in columns.  
aseries[i,j] is the mean over month j for year i.

If you want the monthly values over all the years, just take the mean  
along axis=0

>
>
>> If you have a monthly frequency: just use .convert('A') and compute
>> the stats for each column.
>
> Wouldn't this convert it to an Annual frequency, ie one value per  
> year?

It does, but because you don't precise a func, you end up with a 2D  
array

http://pytseries.sourceforge.net/generated/scikits.timeseries.TimeSeries.convert.html#scikits.timeseries.TimeSeries.convert 


>> The generic idea is indeed to first convert to monthly. You end up
>> with a (12*n, 31) series that you have to reshape
>
> I get the 12*n , ie 12 months * number of years. Where does the 31  
> come from. Does the series still contain the daily values after  
> conversion to monthly?

I was talking about a 'D'aily series to convert to monthly, without  
using a 'func' parameter. You then end up with a 2D series, with 12*n  
rows and 31 columns (because there's at most 31 days per month). Note  
that the last column (day 31) would be masked for Feb, Apr, Jun, Sep,  
Nov.


_______________________________________________
SciPy-user mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-user mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries : how would Iplot (orcalculate) monthly statistics.

Pierre GM-2

On Mar 5, 2009, at 2:49 PM, Dharhas Pothina wrote:

>
> Thank you. This worked great and saved me a lot of time.

Good to hear!


> I'm slowly getting the hang of this package. Sorry for the obvious  
> question but I assume that ma.mean()  etc automatically excludes  
> masked values from the calculation.

Indeed. Most of the time, the np and ma versions of functions behave  
similarly, but it is recommended to stick to the ma versions. Here,  
ma.mean computes the mean by discarding masked values.
_______________________________________________
SciPy-user mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user