Hi,
This a question for those of you who use scikits.timeseries . Given 1015 years of timeseries data how would I plot monthly statistics like max, min, mean, std deviation etc for a year. ie. on the x axis would be the 12 months (Jan,Feb...Dec) and y axis would have the stats. I know how to read in my timeseries and convert it to a monthly frequency but I'm not sure the best way to proceed from here. I guess one way would be to reshape the monthly timeseries array into 12 columns and then calculate the stats on each column but I was wondering if there was a more appropriate method to do this? On another note: shouldn't there be a link on this page (http://scikits.appspot.com/timeseries) to the timeseries home page http://pytseries.sourceforge.net/ . The link on the appspot page (http://pypi.python.org/pypi/scikits.timeseries) doesn't work. thanks  dharhas _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser 
Administrator

Hello
> Given 1015 years of timeseries data how would I plot monthly > statistics like max, min, mean, std deviation etc for a year. Something like: import scikits.timeseries as ts import numpy as np start_data = ts.Date(freq='M', year=1990, month=1) data = np.random.uniform(0, 20, 120) ts_monthly = ts.time_series(data, freq='M', start_date=start_data) aser = ts_monthly.convert('A', func=np.ma.std) => now plot aser? Regards, Timmie P.S.: regarding time series, I have sth. for you . Please pass me your PM. _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser 
In reply to this post by Dharhas Pothina
On Mar 4, 2009, at 11:15 AM, Dharhas Pothina wrote: > > Given 1015 years of timeseries data What frequency ? Monthly ? Daily ? > > I know how to read in my timeseries and convert it to a monthly > frequency but I'm not sure the best way to proceed from here. I > guess one way would be to reshape the monthly timeseries array into > 12 columns and then calculate the stats on each column but I was > wondering if there was a more appropriate method to do this? * If you have a monhtly frequency: just use .convert('A') and compute the stats for each column. * If you have a daily frequency: well, I'm afraid you didn't give enough information: what do you want for each month, what is the expected shape of the output ? Statistics far all the years (eg, the mean for January irrespectively of the year), with an output of shape (12,) ? For each year, with an output of shape (n, 12) and n the nb of years? The generic idea is indeed to first convert to monthly. You end up with a (12*n, 31) series that you have to reshape. However, because you won't need the dates, I strongly advise you to work only on the .series part for the reshaping. > On another note: shouldn't there be a link on this page (http://scikits.appspot.com/timeseries > ) to the timeseries home page http://pytseries.sourceforge.net/ . > The link on the appspot page (http://pypi.python.org/pypi/scikits.timeseries > ) doesn't work. Thanks for reporting. We were already aware of the problem, that shall be fixed when timeseries will be officially released (and then, we'll be able to put it on pypi). _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser 
Original data is on an approximately two week frequency. I'll be reading them in as daily values and converting to monthly. I eventually want to calculate plot monthly values in a particular year vs mean monthly values for all years (ie vs mean for each month irrespective of year) I'll try playing around with the what you and Tim have mentioned. but I'm a bit confused about how some of this works. > If you have a monthly frequency: just use .convert('A') and compute > the stats for each column. Wouldn't this convert it to an Annual frequency, ie one value per year? > The generic idea is indeed to first convert to monthly. You end up > with a (12*n, 31) series that you have to reshape I get the 12*n , ie 12 months * number of years. Where does the 31 come from. Does the series still contain the daily values after conversion to monthly? thanks,  dharhas >>> Pierre GM <[hidden email]> 3/4/2009 2:38 PM >>> On Mar 4, 2009, at 11:15 AM, Dharhas Pothina wrote: > > Given 1015 years of timeseries data What frequency ? Monthly ? Daily ? > > I know how to read in my timeseries and convert it to a monthly > frequency but I'm not sure the best way to proceed from here. I > guess one way would be to reshape the monthly timeseries array into > 12 columns and then calculate the stats on each column but I was > wondering if there was a more appropriate method to do this? * If you have a monhtly frequency: just use .convert('A') and compute the stats for each column. * If you have a daily frequency: well, I'm afraid you didn't give enough information: what do you want for each month, what is the expected shape of the output ? Statistics far all the years (eg, the mean for January irrespectively of the year), with an output of shape (12,) ? For each year, with an output of shape (n, 12) and n the nb of years? The generic idea is indeed to first convert to monthly. You end up with a (12*n, 31) series that you have to reshape. However, because you won't need the dates, I strongly advise you to work only on the .series part for the reshaping. > On another note: shouldn't there be a link on this page (http://scikits.appspot.com/timeseries > ) to the timeseries home page http://pytseries.sourceforge.net/ . > The link on the appspot page (http://pypi.python.org/pypi/scikits.timeseries > ) doesn't work. Thanks for reporting. We were already aware of the problem, that shall be fixed when timeseries will be officially released (and then, we'll be able to put it on pypi). _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser 
Administrator

> Original data is on an approximately two week frequency. I'll be reading them
in as daily values and > converting to monthly. It would be helpful if you could create an example maybe using random data. > > If you have a monthly frequency: just use .convert('A') and compute > > the stats for each column. > > Wouldn't this convert it to an Annual frequency, ie one value per year? Yes, I though this is what you wanted. If not maybe you want to take a look at: scikits.timeseries.extrax.convert_to_annual(series) in http://bazaar.launchpad.net/~vcsimports/scipy/timeseries/annotate/head%3A/scikits/timeseries//extras.py This will align the years. Then you can create an average over the years. Greetings, Timmie _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser 
In reply to this post by Dharhas Pothina
Dharhas,
On Mar 4, 2009, at 4:26 PM, Dharhas Pothina wrote: > Original data is on an approximately two week frequency. I'll be > reading them in as daily values and converting to monthly. OK, you could try 'W', but 'D' is also OK. > I eventually want to calculate plot monthly values in a particular > year vs mean monthly values for all years (ie vs mean for each month > irrespective of year) Then, that's quite easy: First, compute the means per month: >>> mseries = series.convert('M', func=ma.mean) You end up with a (n*12) series. Convert it to annual, but w/o precising a function >>> aseries = mseries.convert('A') aseries is a (n, 12) series: years in rows, months in columns. aseries[i,j] is the mean over month j for year i. If you want the monthly values over all the years, just take the mean along axis=0 > > >> If you have a monthly frequency: just use .convert('A') and compute >> the stats for each column. > > Wouldn't this convert it to an Annual frequency, ie one value per > year? It does, but because you don't precise a func, you end up with a 2D array http://pytseries.sourceforge.net/generated/scikits.timeseries.TimeSeries.convert.html#scikits.timeseries.TimeSeries.convert >> The generic idea is indeed to first convert to monthly. You end up >> with a (12*n, 31) series that you have to reshape > > I get the 12*n , ie 12 months * number of years. Where does the 31 > come from. Does the series still contain the daily values after > conversion to monthly? I was talking about a 'D'aily series to convert to monthly, without using a 'func' parameter. You then end up with a 2D series, with 12*n rows and 31 columns (because there's at most 31 days per month). Note that the last column (day 31) would be masked for Feb, Apr, Jun, Sep, Nov. _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser 
Thank you. This worked great and saved me a lot of time. I'm slowly getting the hang of this package. Sorry for the obvious question but I assume that ma.mean() etc automatically excludes masked values from the calculation.  d >>> Pierre GM <[hidden email]> 3/4/2009 3:46 PM >>> Dharhas, On Mar 4, 2009, at 4:26 PM, Dharhas Pothina wrote: > Original data is on an approximately two week frequency. I'll be > reading them in as daily values and converting to monthly. OK, you could try 'W', but 'D' is also OK. > I eventually want to calculate plot monthly values in a particular > year vs mean monthly values for all years (ie vs mean for each month > irrespective of year) Then, that's quite easy: First, compute the means per month: >>> mseries = series.convert('M', func=ma.mean) You end up with a (n*12) series. Convert it to annual, but w/o precising a function >>> aseries = mseries.convert('A') aseries is a (n, 12) series: years in rows, months in columns. aseries[i,j] is the mean over month j for year i. If you want the monthly values over all the years, just take the mean along axis=0 > > >> If you have a monthly frequency: just use .convert('A') and compute >> the stats for each column. > > Wouldn't this convert it to an Annual frequency, ie one value per > year? It does, but because you don't precise a func, you end up with a 2D array http://pytseries.sourceforge.net/generated/scikits.timeseries.TimeSeries.convert.html#scikits.timeseries.TimeSeries.convert >> The generic idea is indeed to first convert to monthly. You end up >> with a (12*n, 31) series that you have to reshape > > I get the 12*n , ie 12 months * number of years. Where does the 31 > come from. Does the series still contain the daily values after > conversion to monthly? I was talking about a 'D'aily series to convert to monthly, without using a 'func' parameter. You then end up with a 2D series, with 12*n rows and 31 columns (because there's at most 31 days per month). Note that the last column (day 31) would be masked for Feb, Apr, Jun, Sep, Nov. _______________________________________________ SciPyuser mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/scipyuser _______________________________________________ SciPyuser mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipyuser 
On Mar 5, 2009, at 2:49 PM, Dharhas Pothina wrote: > > Thank you. This worked great and saved me a lot of time. Good to hear! > I'm slowly getting the hang of this package. Sorry for the obvious > question but I assume that ma.mean() etc automatically excludes > masked values from the calculation. Indeed. Most of the time, the np and ma versions of functions behave similarly, but it is recommended to stick to the ma versions. Here, ma.mean computes the mean by discarding masked values. _______________________________________________ SciPyuser mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipyuser 
Free forum by Nabble  Edit this page 