Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Choosing log file destination in logging configuration file

Reply
Thread Tools

Choosing log file destination in logging configuration file

 
 
guybenron@gmail.com
Guest
Posts: n/a
 
      04-22-2008
Hey,

I have a sort of petty-neurotic question, I'm kinda pedantic and like
to keep my log directories clean, and this thing is bothering me to
the point of actually posting a question (couldn't find any post about
this..).

My application has two types of components: daemons and regular ones
that are run every on crontab. Because the log structure is very
similar, I want both to use the same logging configuration file.

For clarity, I want all of the files to have the date as part of the
filename.
The TimedRotatingFileHandler has it built in (once it rotates), and I
found a dirty hack to make it work for the FileHandler (see shameful
code below). Basically I'm all set - have the daemons use the rotating
handler, since they're, um, daemonic and on all the time, and have the
regular components use the regular handler, using the dirty hack to
insert the date into the filename.

So far so good. In the relevant applications, the code looks something
like this:
logging.config.fileConfig('log.ini')
logger = logging.getLogger('log.regular') <or> logger =
logging.getLogger('log.daemonic')
... and start logging.

The thorn in my side is that after the fileConfig call, BOTH handlers
are instantiated, meaning both types of files are created for every
component, even if I don't need it: component.log (for the rotating
handler) and compenent.date.log (for the regular file handler).

...So finally, here's my question:
Apart from splitting the logging configuration into two separate
files, is there any way to NOT create the file until you actually use
it?

Here's the logging configuration file for reference.. Thanks !

[loggers]
keys=root,regular,daemonic

[handlers]
keys=fileHandler,consoleHandler,timedRotatingFileH andler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_regular]
level=INFO
handlers=fileHandler,consoleHandler
propagate=0
qualname=log.regular

[logger_daemonic]
level=INFO
handlers=timedRotatingFileHandler,consoleHandler
propagate=0
qualname=log.daemonic

[handler_timedRotatingFileHandler]
class=handlers.TimedRotatingFileHandler
level=INFO
formatter=simpleFormatter
args=('mylog.log', 'midnight')

[handler_fileHandler]
class=FileHandler
level=INFO
formatter=simpleFormatter
; sorry about the grossness
args=('mylog.' + str(time.localtime().tm_year) + '_' +
str(time.localtime().tm_mon).zfill(2) + '_' +
str(time.localtime().tm_mday).zfill(2) + '.log', 'a')

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[formatter_simpleFormatter]
format=%(asctime)s - %(filename)s - %(name)s - %(levelname)s - %
(message)s
 
Reply With Quote
 
 
 
 
Miki
Guest
Posts: n/a
 
      04-22-2008
Hello,

> So far so good. In the relevant applications, the code looks something
> like this:
> logging.config.fileConfig('log.ini')
> logger = logging.getLogger('log.regular') <or> logger =
> logging.getLogger('log.daemonic')
> .. and start logging.
>
> The thorn in my side is that after the fileConfig call, BOTH handlers
> are instantiated, meaning both types of files are created for every
> component, even if I don't need it: component.log (for the rotating
> handler) and compenent.date.log (for the regular file handler).
>
> ..So finally, here's my question:
> Apart from splitting the logging configuration into two separate
> files, is there any way to NOT create the file until you actually use
> it?

You can generate the .ini file on the fly and then load it:
>>> log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
>>> atexit.register(lambda: remove(log_ini))
>>> logging.config.fileConfig(log_ini)


HTH,
--
Miki <(E-Mail Removed)>
http://pythonwise.blogspot.com
 
Reply With Quote
 
 
 
 
guybenron@gmail.com
Guest
Posts: n/a
 
      04-22-2008
On Apr 22, 12:57 pm, Miki <(E-Mail Removed)> wrote:
> Hello,
>
>
>
> > So far so good. In the relevant applications, the code looks something
> > like this:
> > logging.config.fileConfig('log.ini')
> > logger = logging.getLogger('log.regular') <or> logger =
> > logging.getLogger('log.daemonic')
> > .. and start logging.

>
> > The thorn in my side is that after the fileConfig call, BOTH handlers
> > are instantiated, meaning both types of files are created for every
> > component, even if I don't need it: component.log (for the rotating
> > handler) and compenent.date.log (for the regular file handler).

>
> > ..So finally, here's my question:
> > Apart from splitting the logging configuration into two separate
> > files, is there any way to NOT create the file until you actually use
> > it?

>
> You can generate the .ini file on the fly and then load it:
>
> >>> log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
> >>> atexit.register(lambda: remove(log_ini))
> >>> logging.config.fileConfig(log_ini)

>
> HTH,
> --
> Miki <(E-Mail Removed)>http://pythonwise.blogspot.com



I think it misses the point of having a file to config..
It looks as if there's no solution. A peek at the logging module shows
this:
klass = cp.get(sectname, "class")
...
klass = eval(klass, vars(logging))
args = cp.get(sectname, "args")
args = eval(args, vars(logging))
h = apply(klass, args)

Bah. I'll have to split them into two configuration files (or subclass
and have the respective file and rotating handlers lazy evaluate until
the actual log call is made).
Thanks though.
 
Reply With Quote
 
Vinay Sajip
Guest
Posts: n/a
 
      04-23-2008
On Apr 22, 9:48 pm, (E-Mail Removed) wrote:
> On Apr 22, 12:57 pm, Miki <(E-Mail Removed)> wrote:
>
>
>
> > Hello,

>
> > > So far so good. In the relevant applications, the code looks something
> > > like this:
> > >logging.config.fileConfig('log.ini')
> > > logger =logging.getLogger('log.regular') <or> logger =
> > >logging.getLogger('log.daemonic')
> > > .. and startlogging.

>
> > > The thorn in my side is that after the fileConfig call, BOTH handlers
> > > are instantiated, meaning both types of files are created for every
> > > component, even if I don't need it: component.log (for the rotating
> > > handler) and compenent.date.log (for the regular file handler).

>
> > > ..So finally, here's my question:
> > > Apart from splitting theloggingconfiguration into two separate
> > > files, is there any way to NOT create the file until you actually use
> > > it?

>
> > You can generate the .ini file on the fly and then load it:

>
> > >>> log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
> > >>> atexit.register(lambda: remove(log_ini))
> > >>>logging.config.fileConfig(log_ini)

>
> > HTH,
> > --
> > Miki <(E-Mail Removed)>http://pythonwise.blogspot.com

>
> I think it misses the point of having a file to config..
> It looks as if there's no solution. A peek at theloggingmodule shows
> this:
> klass = cp.get(sectname, "class")
> ...
> klass = eval(klass, vars(logging))
> args = cp.get(sectname, "args")
> args = eval(args, vars(logging))
> h = apply(klass, args)
>
> Bah. I'll have to split them into two configuration files (or subclass
> and have the respective file and rotating handlers lazy evaluate until
> the actual log call is made).
> Thanks though.


Not sure if it's any help - the SVN version contains a new optional
"delay" parameter for FileHandler and subclasses (including the
rotating handlers) which delays opening the file until there's a need
to, i.e. when an emit() call occurs.

Regards,

Vinay Sajip
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Logging exception with java.util.logging.Logger#log() to file only Royan Java 1 04-01-2008 09:42 PM
can PIX 515E log traffic by internal destination host? jason Cisco 2 07-27-2005 03:05 PM
Choosing a database to work with XML logging files. David Collier XML 0 07-26-2004 02:45 PM
Destination not reachable until destination pings source! PIX501 Dave Cisco 0 02-27-2004 06:15 PM
[java.util.logging] logging only to _one_ file Stefan Siegl Java 0 08-27-2003 12:29 PM



Advertisments