Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   line duplication using logging to file (http://www.velocityreviews.com/forums/t398027-line-duplication-using-logging-to-file.html)

seb 01-09-2007 08:06 AM

line duplication using logging to file
 
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.

*****************************************
The very simple "logging program ":
*****************************************
import logging, logging.handlers
import time

def write_log(level, message):
# Utilisation de l'API pour le Handler global
print "time.asctime()",time.asctime(),"received level=",level,"message
=",message
nom_logger="main_log_file"
logger=logging.getLogger(nom_logger)
logger.setLevel(logging.DEBUG)
prefix = "pix_main_log_file"
#fh=logging.handlers.RotatingFileHandler(prefix + "_log.txt", 'a',
1000000,10)
fh=logging.FileHandler("main_log.txt")
fh.setLevel(logging.DEBUG)
#formater = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s
- %(message)s")
formater = logging.Formatter("%(asctime)s - %(levelname)s -
%(message)s")
fh.setFormatter(formater)
logger.addHandler(fh)
#logger.info("***************************** debut")


#message = main(url,conf_file,logger)
#message="hello seb"
if str(level).lower() == "info" :
print "logger info"
logger.info(str(message))
elif str(level).lower() =="error":
print "logger error"
logger.error(str(message))
elif str(level).lower()=="warning" :
print "logger warning"
logger.warning(str(message))
elif str(level).lower() =="critical":
print "logger critical"
logger.critical(str(message))
elif str(level).lower() == "exception":
print "logger exception"
logger.exception(str(message))
else :
logger.info("niveau inconnu "+str(message))

print "_________",message
#print dir(logger)
return

*************************************************
Example of the log file with duplicated line.
**************************************************

2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openC OM1
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:51:26,562 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openC OM1
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret


**************************
Program calling the log :
***************************
import appel_log
......

message = "something"
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
appel_log.write_log("info",nom_function_actuelle+m essage)
.................


Vinay Sajip 01-09-2007 10:07 AM

Re: line duplication using logging to file
 
seb wrote:

> Hi, I am writing to a file some basic information using the logging
> module. It is working but in the log file some line are printed
> several time. I had put some print debugging messages in the logging
> function (so they appear on the consile) and they are called once only.
> Obviously there is some understantding of the logging module that I am
> missing.
>
> My simple logging program (see below) is called by several processes.
> In this way I can collect the information from various sources (and not
> use the network enabled logging module)
>
> I am using python 2.4 on WinXP SP2.
>
> Do you have any idea ? Thanks in advance.
>


I copied your two scripts into "appel_main.py" and "appel_log.py" and
when I run, I get a single line in main_log.txt:

2007-01-09 10:01:42,766 - INFO -C:\temp\appel_main.py___?something

On the console is printed:

time.asctime() Tue Jan 09 10:04:25 2007 received level= info message=
C:\temp\appel_main.py___?something
logger info
_________ C:\temp\appel_main.py___?something

I didn't see any repetitions. Perhaps it's to do with multiple
processes in your environment?

If you call write_log several times you will add a handler several
times - this is not a normal pattern of use. You should separate setup
of Loggers/Handlers/Formatters from actually logging events via
debug(), info() etc.

Best regards,

Vinay Sajip


Paul McGuire 01-09-2007 12:05 PM

Re: line duplication using logging to file
 
"seb" <sebastien.thur@laposte.net> wrote in message
news:1168329975.491514.282000@11g2000cwr.googlegro ups.com...
> Hi, I am writing to a file some basic information using the logging
> module. It is working but in the log file some line are printed
> several time. I had put some print debugging messages in the logging
> function (so they appear on the consile) and they are called once only.
> Obviously there is some understantding of the logging module that I am
> missing.
>
> My simple logging program (see below) is called by several processes.
> In this way I can collect the information from various sources (and not
> use the network enabled logging module)
>
> I am using python 2.4 on WinXP SP2.
>
> Do you have any idea ? Thanks in advance.
>
> Seb.
>


A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul


(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,rest OfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word (nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts



seb 01-09-2007 12:52 PM

Re: line duplication using logging to file
 
Hi,

Thanks for the help.
Meanwhile I have written the logging function from scratch and it works
without the multiple lines.

This means that the multiple line copy is not due to the multiple
processes (or thread) trying to access the log file but to something
else.

Thanks.
Sebastien.

the new function :


import logging
import time

def write_log(level, message):
nom_logger="main_log_file.txt"
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
filename=nom_logger,
filemode='w')

if str(level).lower() == "info" :
logging.info(str(message))
elif str(level).lower() =="error":
logging.error(str(message))
elif str(level).lower()=="warning" :
logging.warning(str(message))
elif str(level).lower() =="critical":
logging.critical(str(message))
elif str(level).lower() == "exception":
logging.exception(str(message))
else :
logging.INFO(str(message))

return







Paul McGuire a écrit :
> "seb" <sebastien.thur@laposte.net> wrote in message
> news:1168329975.491514.282000@11g2000cwr.googlegro ups.com...
> > Hi, I am writing to a file some basic information using the logging
> > module. It is working but in the log file some line are printed
> > several time. I had put some print debugging messages in the logging
> > function (so they appear on the consile) and they are called once only.
> > Obviously there is some understantding of the logging module that I am
> > missing.
> >
> > My simple logging program (see below) is called by several processes.
> > In this way I can collect the information from various sources (and not
> > use the network enabled logging module)
> >
> > I am using python 2.4 on WinXP SP2.
> >
> > Do you have any idea ? Thanks in advance.
> >
> > Seb.
> >

>
> A quick tally of log messages by timestamp and comment gives this data:
>
> ('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
> ('2007-01-08 18:26:19,578', '___run____thread lance') : 2
> ('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
> ('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
> ('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
> ('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
> ('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
> ('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
> ('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
> ('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
> ('2007-01-09 08:51:26,733', '___run____thread lance') : 2
> ('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
> ('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
> ('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
> ('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
> ('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
> ('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
> ('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
> ('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1
>
> Does this suggest anything to you?
>
> -- Paul
>
>
> (BTW, here is the pyparsing program I used to do this analysis)
>
> from pyparsing import
> Word,nums,Combine,alphas,oneOf,Literal,SkipTo,rest OfLine
>
> # create pyparsing grammar definition for a log line
> date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
> time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word (nums,exact=2)+ \
> ','+Word(nums,exact=3)
> timestamp = Combine(date + ' ' + time)
> severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
> data
> backslash = Literal("\\")
> fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
> ".py")
> logline = ( timestamp.setResultsName("timestamp") + "-" +
> severity.setResultsName("severity") + "-" +
> fileref.setResultsName("fileref") +
> restOfLine.setResultsName("comment") )
>
> # create list of ParseResults, with addressable log line elements by results
> name
> logEntries = [ logline.parseString(line) for line in logdata ]
>
> # tally up log lines by timestamp and comment
> tallyByTimestamp = {}
> for entry in logEntries:
> tallyKey = (entry.timestamp, entry.comment)
> tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1
>
> for ts in sorted( tallyByTimestamp.items() ):
> print "%s : %d" % ts




All times are GMT. The time now is 05:43 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.