scikits.timeseries.tsfromtxt & guess

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

scikits.timeseries.tsfromtxt & guess

Timmie
Administrator
Hello,
I first want to stress again that the tsfromtxt in the timeseries scikit is a
real killer function.
Once one has understood the easyness of the "dateconverter" function it becomes
a quick exercise to read in timeseries from ASCII files.

As I am currently predefining a set of dateconverters for frquently used
date-time combinations in different formats, I have the following question:
Is it possible to integrate "ts.extras.guess_freq(dates)" into the function
scikits.timeseries.tsfromtxt?


Currently, I would need to read a file twice: once for guessing the frequency
based on a created list of dates and then read file to create the timeseries.

Ideally, I would like to do:
 


def mydateconverter(year, month, day, hour):
    freq = ts.extras.guess_freq(year, month, day, hour)
    ts_date =  ts.Date(freq, year=int(year), month=int(month), day=int(day))
   
    return ts_date


myts= ts.tsfromtxt(datafile, skiprows=1, names=None,
                      datecols=(1,2,3), guess_freq=True,  
                      dateconverter=mydateconverter)

Or is this already possible and I am just not getting this right?


How can I pass a frequency value to the dateconverter argument?

Like:
def mydateconverter(year, month, day, hour, freq='T'):
    freq = ts.extras.guess_freq(year, month, day, hour)
    ts_date =  ts.Date(freq, year=int(year), month=int(month), day=int(day))
   
    return ts_date

myts= ts.tsfromtxt(datafile, skiprows=1, names=None,
                      datecols=(1,2,3), guess_freq=True,  
                      dateconverter=mydateconverter(freq='H'))

I get this error then:
TypeError: mydateconverter() takes at least 2 non-keyword arguments (0 given)

Thanks in advance for any hints,
Timmie

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries.tsfromtxt & guess

Pierre GM-2
On Jan 4, 2010, at 12:11 PM, Tim Michelsen wrote:
> Hello,
> I first want to stress again that the tsfromtxt in the timeseries scikit is a
> real killer function.

My, thanks a lot

> Once one has understood the easyness of the "dateconverter" function it becomes
> a quick exercise to read in timeseries from ASCII files.
>
> As I am currently predefining a set of dateconverters for frquently used
> date-time combinations in different formats, I have the following question:
> Is it possible to integrate "ts.extras.guess_freq(dates)" into the function
> scikits.timeseries.tsfromtxt?

Probably, but I doubt it'll be very different from the current behavior. See, the dateconverter function transforms a series of strings into a unique Date, independently for each row of the input. You can't guess the frequency of an individual Date, you need several to compare their lags. That means that no matter what, you'll have to reprocess the array. I'd prefer to leave this operation up to the user...

> Currently, I would need to read a file twice: once for guessing the frequency
> based on a created list of dates and then read file to create the timeseries.

I'd first create the time series from the input, then try to guess the frequency from the DateArray

>
> How can I pass a frequency value to the dateconverter argument?
>
> Like:
> def mydateconverter(year, month, day, hour, freq='T'):
>    freq = ts.extras.guess_freq(year, month, day, hour)
>    ts_date =  ts.Date(freq, year=int(year), month=int(month), day=int(day))
>
>    return ts_date
>
> myts= ts.tsfromtxt(datafile, skiprows=1, names=None,
>                      datecols=(1,2,3), guess_freq=True,  
>                      dateconverter=mydateconverter(freq='H'))
>
> I get this error then:
> TypeError: mydateconverter() takes at least 2 non-keyword arguments (0 given)

Please send a small example of datafile so that I can test wht goes wrong. If I have to guess: the line `dateconverter=mydateconverter(freq='H')` forces a call to mydateconverter without any argument (but for he frequency). Of course, that won't fly.
What you want is to have `mydateconverter(freq='H')` callable. You should probably create a class that takes a frequency as instantiation input and that has a __call__ method, something like:

class myconverter(object)
    def __init__(freq='D'):
        self.freq=freq
    def __call__(self, y,m,d,h):
        return ts.Date(self.freq, year=int(y),month=int(m),day=int(day),hour=int(h))

That way, myconverter(freq='T') becomes a valid function (you can call it).
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries.tsfromtxt & guess

Timmie
Administrator
>> I first want to stress again that the tsfromtxt in the timeseries scikit is a
>> real killer function.
>
> My, thanks a lot
Yes, you may remember all my questions (still at the beginning of my
scipy learning curve) on the data loading and creation of masked time
series...
This is now all obsolete. And as I receive data (logger) in wicked
formats not counting from 0-23 but rather 1-24, I appreciate the
datconverters which are based on strong datetime manupulations.

> I'd first create the time series from the input, then try to guess the frequency from the DateArray
So you'd recommend to create the timeseries using the userdefined
frequency ('U')
def mydateconverter(year, month, day, hour, freq='U'):
    freq = ts.extras.guess_freq(year, month, day, hour)
    ts_date =  ts.Date(freq, year=int(year), month=int(month), day=int(day))

    return ts_date

and then use guess_freq to assign the correct one?

I want to have the dateconverters in a flexible style only variyng by
input format and clumns used. They should be working regardless of the
frequency (be the data set hourly or minutely).

>
>> How can I pass a frequency value to the dateconverter argument?
>>
>> Like:
>> def mydateconverter(year, month, day, hour, freq='T'):
>>    freq = ts.extras.guess_freq(year, month, day, hour)
>>    ts_date =  ts.Date(freq, year=int(year), month=int(month), day=int(day))
>>
>>    return ts_date
>>
>> myts= ts.tsfromtxt(datafile, skiprows=1, names=None,
>>                      datecols=(1,2,3), guess_freq=True,  
>>                      dateconverter=mydateconverter(freq='H'))
>>
>> I get this error then:
>> TypeError: mydateconverter() takes at least 2 non-keyword arguments (0 given)
>
> Please send a small example of datafile so that I can test wht goes wrong. If I have to guess: the line `dateconverter=mydateconverter(freq='H')` forces a call to mydateconverter without any argument (but for he frequency). Of course, that won't fly.
> What you want is to have `mydateconverter(freq='H')` callable. You should probably create a class that takes a frequency as instantiation input and that has a __call__ method, something like:
>
> class myconverter(object)
>     def __init__(freq='D'):
>         self.freq=freq
>     def __call__(self, y,m,d,h):
>         return ts.Date(self.freq, year=int(y),month=int(m),day=int(day),hour=int(h))
>
> That way, myconverter(freq='T') becomes a valid function (you can call it).
Thanks. I will try this way.

Best regards,
Timmie

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: scikits.timeseries.tsfromtxt & guess

Pierre GM-2
On Jan 4, 2010, at 3:25 PM, Tim Michelsen wrote:

>> I'd first create the time series from the input, then try to guess the frequency from the DateArray
> So you'd recommend to create the timeseries using the userdefined
> frequency ('U')
> def mydateconverter(year, month, day, hour, freq='U'):
>    freq = ts.extras.guess_freq(year, month, day, hour)
>    ts_date =  ts.Date(freq, year=int(year), month=int(month), day=int(day))
>
>    return ts_date
>
> and then use guess_freq to assign the correct one?

Basically, yes. Note that guess_freq is only for convenience, it might not be fool-proof...


> I want to have the dateconverters in a flexible style only variyng by
> input format and clumns used. They should be working regardless of the
> frequency (be the data set hourly or minutely).

Well, you could define a converter class that takes freq as input and test in the __call__ for the value of the freq. You could have a variable nb of inputs in __call__ and test for the nb of parameters (year, month, day...). It won't be as efficient as defining a specific converter for your data, though...

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user