[SciPy-User] Is there existing code to log-with-bells-on for runtime algorithm diagnostics?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] Is there existing code to log-with-bells-on for runtime algorithm diagnostics?

Rob Clewley
Hi,

I'm in need of a system for logging the step-wise results and
diagnostic metadata about a python function implementation of an
algorithm that I'm developing. The specific algorithm is not of great
consequence except that it's for scientific computing and may produce
large (e.g., '00s or maybe '000s, but not "big data" scale) amounts of
intermediate numerical data that can be complex to understand when
debugging its progress.

In fact, I'm trying to build a general purpose tool for exploring the
inner workings of numerical algorithms for teaching and learning
purposes, e.g. for graduate student training or for figuring out
parameter choices in difficult applications.

I want to be able to insert commands inside of the loops that log
certain variable states, completed stages of the algorithm (assume
it's hierarchical), text warnings, and possibly even 'pointers' to
graphical output objects of some of the intermediate data (e.g.
matplotlib object handles for lines, points). Then I can trace the
work done afterwards or step through the process in an IDE debugger
and make interactive calls to access recent steps, plot certain
relationships in the current state, etc. The basic logger's "levels"
of output don't really apply here. I at least want categories, if not
hierarchical sub-categories.

I don't think the built-in logger is sophisticated enough for this,
being a flat record of freeform text AFAIU, but the API looks
appealing.

I'm considering an in-memory sqlite DB to store structured records at
any logged step, and an accompanying dictionary to store references to
any python object metadata, keyed by a unique ID in the DB log. Then
I'd write an API for it that resembles the logger's.

It's not super hard for me to write my own thing here, but I'm
wondering if anyone has come across any existing solutions in this
vein, or has any advice before I go further in designing a solution? I
can't really believe that no-one has attempted this before, but it's
been really hard to find any existing work through online search.

Thanks,
Rob
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Is there existing code to log-with-bells-on for runtime algorithm diagnostics?

Thomas Kluyver-2
On 21 April 2015 at 12:02, Rob Clewley <[hidden email]> wrote:
The basic logger's "levels"
of output don't really apply here. I at least want categories, if not
hierarchical sub-categories.

Python's built-in logging has a hierarchy of loggers according to the name - you can set up log handling for the logger named 'foo', and things from the logger 'foo.bar' will go to that. The convention is to instantiate each logger with a module name, so you can configure logging by package, but I think you could use it with any hierarchy.

It is text-only, as far as I know, but it might save you some work.

Thomas

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Is there existing code to log-with-bells-on for runtime algorithm diagnostics?

Robert Kern-2
In reply to this post by Rob Clewley
On Tue, Apr 21, 2015 at 8:02 PM, Rob Clewley <[hidden email]> wrote:

>
> Hi,
>
> I'm in need of a system for logging the step-wise results and
> diagnostic metadata about a python function implementation of an
> algorithm that I'm developing. The specific algorithm is not of great
> consequence except that it's for scientific computing and may produce
> large (e.g., '00s or maybe '000s, but not "big data" scale) amounts of
> intermediate numerical data that can be complex to understand when
> debugging its progress.
>
> In fact, I'm trying to build a general purpose tool for exploring the
> inner workings of numerical algorithms for teaching and learning
> purposes, e.g. for graduate student training or for figuring out
> parameter choices in difficult applications.

The term you want to search for is "structured logging".

http://www.structlog.org/en/stable/
http://eliot.readthedocs.org/en/stable/
https://twiggy.readthedocs.org/en/latest/logging.html#structured-logging
http://netlogger.lbl.gov/

--
Robert Kern

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Is there existing code to log-with-bells-on for runtime algorithm diagnostics?

Rob Clewley
All of these ideas and links are very helpful, thank you!
-Rob
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Is there existing code to log-with-bells-on for runtime algorithm diagnostics?

Rob Clewley
In reply to this post by Robert Kern-2
Just to follow up on this thread, for interested readers' future reference...

On Tue, Apr 21, 2015 at 4:22 PM, Robert Kern <[hidden email]> wrote:

> On Tue, Apr 21, 2015 at 8:02 PM, Rob Clewley <[hidden email]> wrote:
>> In fact, I'm trying to build a general purpose tool for exploring the
>> inner workings of numerical algorithms for teaching and learning
>> purposes, e.g. for graduate student training or for figuring out
>> parameter choices in difficult applications.
>
> The term you want to search for is "structured logging".
>
> http://www.structlog.org/en/stable/
> http://eliot.readthedocs.org/en/stable/
> https://twiggy.readthedocs.org/en/latest/logging.html#structured-logging
> http://netlogger.lbl.gov/
>

I posted a new blog entry about my prototypical diagnosis and
visualization tools for python numerical algorithms, built over
matplotlib:
http://robclewley.github.io/logging-and-diagnostic-tools-for-numeric-python-algorithms/

They utilize structlog, and I hooked up a noSQL DB (tinydb) to the
logging to enable search capabilities of the log post-mortem. Comments
and PRs most welcome.

Thanks to everyone for the pointers.
-Rob
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user