code structure advise for a model

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

code structure advise for a model

mdekauwe
Hi,

I am translating some c++ code to python and just wanted to ask some advise on structure. The original has everything declared globally and nothing passed via function (I assume, but don't know, that this isn't just standard c++ practice!). So given this, I have a pretty much clean slate as I can't quite just copy the functions over. I was thinking something like this

class Params:

    def __init__(self, fname):
        self.set_inital_condtions()
        self.read_input_file(fname)

    def set_inital_conditons(self):    
        self.some_parm = 0.0


    def read_input_file(fname):
       
        #read file, change initial params if specified


then I thought I could pass this as an object to the model class

class Model(Params):

    def __init__(self):
        # blah

    def some_func(self):
         if (Params.some_param == something):
             foo

OR this just a very bad way to structure it?

The other thing I can't decide on is how to pass the parameters and variables through the class. So because of the way the original is written (everything is global), I could just inherit things, but it does means there is a lot of self. syntax. So I wondered if it might be better to pass things as function arguments? Any thoughts? I am also half considering other users from non-python backgrounds and what might seem very alien (syntax) to them.

thanks in advance

(ps. I am cross posting this on comp.lang.python as I am not sure where is more appropriate).
Reply | Threaded
Open this post in threaded view
|

Re: [SciPy-user] code structure advise for a model

David Cournapeau-3
Hi,

On 02/04/2011 11:42 AM, mdekauwe wrote:

>
> Hi,
>
> I am translating some c++ code to python and just wanted to ask some advise
> on structure. The original has everything declared globally and nothing
> passed via function (I assume, but don't know, that this isn't just standard
> c++ practice!). So given this, I have a pretty much clean slate as I can't
> quite just copy the functions over. I was thinking something like this
>
> class Params:
>
>      def __init__(self, fname):
>          self.set_inital_condtions()
>          self.read_input_file(fname)
>
>      def set_inital_conditons(self):
>          self.some_parm = 0.0
>
>
>      def read_input_file(fname):
>
>          #read file, change initial params if specified
>

Bit hard to say without more information, but you often use less custom
classes than in a language like C++/java to pass data around.

One rule of a thumb to decide wether you want class or just pass
arguments through list/dict is whether you need your parameters to be
stateful (which is a good thing to avoid everything else being equal).

As for how to use inheritence: inheritence is tricky. One rule which
works almost all the time to decide where B should derive from A is
whether an instance of B can be used whenever an instance of A needs to
be used (this is called the Liskov subsitution principle if you want to
shine in discussions). Another rule is to avoid inheritence if you can:
you should try to use composition instead, because it is more flexible
(design mistakes in term of inheritence are hard to fix because it
impacts all the subclasses of a changed class).

In that aspect, from what information you gave us, I don't see why Model
should inherit from Params.

As far as Params goes, something like the following would work equally well:

def create_params(some_param=0.0):
     data = {"some_param": some_param, "other_param": ...}
     return data

def create_param_from_file(filename):
     read file content, parse it to get data
     return create_params(some_param)

This is simpler, use common code for the parameters creation. Of course,
it may be that your code is much more complicated, in which case a class
may be better. In the latter case, a common idiom is

class Params(object):
     # Class method just means that from_file is a method attached
     # to the class, not instances (~ static methods in java/c++).
     @classmethod
     def from_file(cls, filename):
          read file content and parse it to get data
          return cls(some_param)

     def __init__(self, some_param=0.0):
         self.some_param = some_param

You then use is as follows:

params1 = Params(some_param=1.0)
params2 = Params.from_file("foo.txt")

Using class methods for atlernate constructions is a very common idiom
in python.

cheers,

David
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: [SciPy-user] code structure advise for a model

mdekauwe
That is very helpful thanks.

So I think I will follow what you suggested and set up two functions e.g.

def setup_initial_conditions():
    params = {"cat": 1, "mouse": 2 ...etc}
    return params

def read_file(params, fname):
    # read file and replace any relevant values in params
    return params

I am still going to use a class for my model as it has a lot of functions that I think tie nicely together

class Model:

    def __init__(self, fname='x.dat'):
        self.fname = fname

    def main(self):
        params = setup_initial_conditions():
        params = read_file(params, self.fname)

    def work_out_something(self):
         etc

I have a series of "pools" which go up and down through time and I will have these as inherited and pass the rest of the model args around the code. I think perhaps based on what you suggested this sounds more sensible?

thanks

David Cournapeau-3 wrote
Hi,

On 02/04/2011 11:42 AM, mdekauwe wrote:
>
> Hi,
>
> I am translating some c++ code to python and just wanted to ask some advise
> on structure. The original has everything declared globally and nothing
> passed via function (I assume, but don't know, that this isn't just standard
> c++ practice!). So given this, I have a pretty much clean slate as I can't
> quite just copy the functions over. I was thinking something like this
>
> class Params:
>
>      def __init__(self, fname):
>          self.set_inital_condtions()
>          self.read_input_file(fname)
>
>      def set_inital_conditons(self):
>          self.some_parm = 0.0
>
>
>      def read_input_file(fname):
>
>          #read file, change initial params if specified
>

Bit hard to say without more information, but you often use less custom
classes than in a language like C++/java to pass data around.

One rule of a thumb to decide wether you want class or just pass
arguments through list/dict is whether you need your parameters to be
stateful (which is a good thing to avoid everything else being equal).

As for how to use inheritence: inheritence is tricky. One rule which
works almost all the time to decide where B should derive from A is
whether an instance of B can be used whenever an instance of A needs to
be used (this is called the Liskov subsitution principle if you want to
shine in discussions). Another rule is to avoid inheritence if you can:
you should try to use composition instead, because it is more flexible
(design mistakes in term of inheritence are hard to fix because it
impacts all the subclasses of a changed class).

In that aspect, from what information you gave us, I don't see why Model
should inherit from Params.

As far as Params goes, something like the following would work equally well:

def create_params(some_param=0.0):
     data = {"some_param": some_param, "other_param": ...}
     return data

def create_param_from_file(filename):
     read file content, parse it to get data
     return create_params(some_param)

This is simpler, use common code for the parameters creation. Of course,
it may be that your code is much more complicated, in which case a class
may be better. In the latter case, a common idiom is

class Params(object):
     # Class method just means that from_file is a method attached
     # to the class, not instances (~ static methods in java/c++).
     @classmethod
     def from_file(cls, filename):
          read file content and parse it to get data
          return cls(some_param)

     def __init__(self, some_param=0.0):
         self.some_param = some_param

You then use is as follows:

params1 = Params(some_param=1.0)
params2 = Params.from_file("foo.txt")

Using class methods for atlernate constructions is a very common idiom
in python.

cheers,

David
_______________________________________________
SciPy-User mailing list
SciPy-User@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: [SciPy-user] code structure advise for a model

Chris Barker - NOAA Federal
In reply to this post by David Cournapeau-3
On 2/3/11 8:42 PM, David wrote:
> As for how to use inheritance: inheritance is tricky. One rule which
> works almost all the time to decide where B should derive from A is
> whether an instance of B can be used whenever an instance of A needs to
> be used (this is called the Liskov substitution principle if you want to
> shine in discussions).

There is also the classic way of thinking about OO:

use inheritance for a "is a" relationship:

a cat "is a" mammal.

> In that aspect, from what information you gave us, I don't see why Model
> should inherit from Params.

let's see: A Model "is a" set of parameters -- nope!

Another common relationship is the "has a" relationship:

A Model "has a" set of parameters -- yup!

So that means that your model would have a Params instance as an attribute.

> def create_params(some_param=0.0):
>       data = {"some_param": some_param, "other_param": ...}
>       return data

yup -- a simple dict may be all you need.

-Chris

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[hidden email]
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: [SciPy-user] code structure advise for a model

mdekauwe
OK given the above perhaps my latest implementation is wrong then?

So to recap I have a model, lets say it predicts the photosynthesis by a plant. There are some parameters which are read in from a file, for example size of leaf, efficiency at converting sunlight etc. So my current implementation is to group all the functions of the model in one class and pass the parameters to this class.

(1).

Based on previous suggestions I used ConfigParser to read a file containing various model parameters into a set of different dictionaries which I then passed around the code (for example control parameters: e.g. out_fname, log_fname, model_number and initial model parameters: eg. leaf size)

However I was finding using the dictionary syntax made the code a little hard to read, so now I have packaged these dictionaries into classes

e.g.

class Dict2Class:
        def __init__(self, d):
                self.__dict__ = d

so now instead of control_param['logfile']

I can just do control_param.logfile.

Does that sound OK? I think it makes the code easier to read personally, but perhaps this is not great?

(2).

In line with the example the model has a set of "pools" (values) that get updated during every loop iterations (for example over all the days in the year). These "pools", for example the total amount of leaves on a tree are required throughout different model functions. I had toyed with inheriting these, but then I think it is hard to follow exactly where they are being adjusted. So I wondered if I could take a similar approach to (1) and declare and empty class and update this when I read the initial conditions file and then pass the pools object around between the model methods.

e.g.

class ModelPools:
    pass

cpools = ModelPools()
cpools.total_leaves = ini_cond['total_leaves']

so a general code structure now looks like this

def read_confile(confname, section, data_type):
    """
    read the config file with various model parameters
    into different dictionary, broken down by section
    """
    return dict


class Dict2Class:
        def __init__(self, d):
                self.__dict__ = d

class ModelPools:
    """ fill when initial conditions file is read    
    pass


class Model:
   
    def __init__(self, config_fname, initialcond_fname):
       
        self.config_fname = config_fname
        self.initialcond_fname = initialcond_fname
       
 
    def main(self):
       
        # read the driving data in
        met_data = met_data = np.loadtxt(self.met_fname, comments='#')
       
        # read the configuration file, keep params grouped in dictionaries
        ini_cond = Dict2Class(rc.read_confile(self.initialcond_fname, 'ini', 'float'))
        photo_params = Dict2Class(rc.read_confile(self.config_fname, 'prodn', 'float'))
        control_params = Dict2Class(rc.read_confile(self.config_fname, 'control', 'float'))
       

        # setup pools structure
        cpools = ModelPools()
        cpools.total_leaves = ini_cond['total_leaves']
       
        self.call_some_func(cpools, control_params, photo_params)

thanks again.

Christopher Barker wrote
On 2/3/11 8:42 PM, David wrote:
> As for how to use inheritance: inheritance is tricky. One rule which
> works almost all the time to decide where B should derive from A is
> whether an instance of B can be used whenever an instance of A needs to
> be used (this is called the Liskov substitution principle if you want to
> shine in discussions).

There is also the classic way of thinking about OO:

use inheritance for a "is a" relationship:

a cat "is a" mammal.

> In that aspect, from what information you gave us, I don't see why Model
> should inherit from Params.

let's see: A Model "is a" set of parameters -- nope!

Another common relationship is the "has a" relationship:

A Model "has a" set of parameters -- yup!

So that means that your model would have a Params instance as an attribute.

> def create_params(some_param=0.0):
>       data = {"some_param": some_param, "other_param": ...}
>       return data

yup -- a simple dict may be all you need.

-Chris

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov
_______________________________________________
SciPy-User mailing list
SciPy-User@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: [SciPy-user] code structure advise for a model

Yung-Yu Chen-2
On Mon, Feb 7, 2011 at 00:27, mdekauwe <[hidden email]> wrote:

OK given the above perhaps my latest implementation is wrong then?

So to recap I have a model, lets say it predicts the photosynthesis by a
plant. There are some parameters which are read in from a file, for example
size of leaf, efficiency at converting sunlight etc. So my current
implementation is to group all the functions of the model in one class and
pass the parameters to this class.

(1).

Based on previous suggestions I used ConfigParser to read a file containing
various model parameters into a set of different dictionaries which I then
passed around the code (for example control parameters: e.g. out_fname,
log_fname, model_number and initial model parameters: eg. leaf size)

However I was finding using the dictionary syntax made the code a little
hard to read, so now I have packaged these dictionaries into classes

e.g.

class Dict2Class:
       def __init__(self, d):
               self.__dict__ = d

so now instead of control_param['logfile']

I can just do control_param.logfile.


For your use case, code like:

class AttributeDict(dict):
    def __getattr__(self, name):
        return self[name]
    def __setattr__(self, name, value):
        if name in self:
            self[name] = value
        else:
            super(AttributeDict, self).__setattr__(name, value)

should be more straightforward.  What you wanted is a dictionary that supports accessing its content through attribute access (x.y).  What you did is replacing the name space of an object.  Semantically it's not what you want.
 
Does that sound OK? I think it makes the code easier to read personally, but
perhaps this is not great?


It should be OK.  But overriding the name space of an object, i.e., __dict__, could surprise you when you want further extension to your class.
 
(2).

[deleted]

with regards,
Yung-Yu Chen

--
Yung-Yu Chen
PhD candidate of Mechanical Engineering
The Ohio State University, Columbus, Ohio
+1 (614) 859 2436


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: [SciPy-user] code structure advise for a model

mdekauwe
Hi,

thanks if you could explain that a little further it would be appreciated.

Martin

Yung-Yu Chen-2 wrote
On Mon, Feb 7, 2011 at 00:27, mdekauwe <mdekauwe@gmail.com> wrote:

>
> OK given the above perhaps my latest implementation is wrong then?
>
> So to recap I have a model, lets say it predicts the photosynthesis by a
> plant. There are some parameters which are read in from a file, for example
> size of leaf, efficiency at converting sunlight etc. So my current
> implementation is to group all the functions of the model in one class and
> pass the parameters to this class.
>
> (1).
>
> Based on previous suggestions I used ConfigParser to read a file containing
> various model parameters into a set of different dictionaries which I then
> passed around the code (for example control parameters: e.g. out_fname,
> log_fname, model_number and initial model parameters: eg. leaf size)
>
> However I was finding using the dictionary syntax made the code a little
> hard to read, so now I have packaged these dictionaries into classes
>
> e.g.
>
> class Dict2Class:
>        def __init__(self, d):
>                self.__dict__ = d
>
> so now instead of control_param['logfile']
>
> I can just do control_param.logfile.
>
>
For your use case, code like:

class AttributeDict(dict):
    def __getattr__(self, name):
        return self[name]
    def __setattr__(self, name, value):
        if name in self:
            self[name] = value
        else:
            super(AttributeDict, self).__setattr__(name, value)

should be more straightforward.  What you wanted is a dictionary that
supports accessing its content through attribute access (x.y).  What you did
is replacing the name space of an object.  Semantically it's not what you
want.


> Does that sound OK? I think it makes the code easier to read personally,
> but
> perhaps this is not great?
>
>
It should be OK.  But overriding the name space of an object, i.e.,
__dict__, could surprise you when you want further extension to your class.


> (2).
>
> [deleted]
>

with regards,
Yung-Yu Chen

--
Yung-Yu Chen
PhD candidate of Mechanical Engineering
The Ohio State University, Columbus, Ohio
+1 (614) 859 2436
http://solvcon.net/yyc/

_______________________________________________
SciPy-User mailing list
SciPy-User@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user