Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: parse date/time from a log entry with only strftime (and noregexen)

Reply
Thread Tools

Re: parse date/time from a log entry with only strftime (and noregexen)

 
 
Gabriel Genellina
Guest
Posts: n/a
 
      02-03-2009
En Tue, 03 Feb 2009 11:52:07 -0200, Simon Mullis <(E-Mail Removed)>
escribió:

> I'm writing a script to help with analyzing log files timestamps and
> have a
> very specific question on which I'm momentarily stumped....
>
> I'd like the script to support multiple log file types, so allow a
> strftime
> format to be passed in as a cli switch (default is %Y-%m-%d %H:%M:%S).


>>>> import datetime
>>>> p = datetime.datetime.strptime("2008-07-23 12:18:28 this is the

> remainder of the log line that I do not care about", "%Y-%m-%d %H:%M:%S")
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/opt/local/lib/python2.5/_strptime.py", line 333, in strptime
> data_string[found.end():])
> ValueError: unconverted data remains: this is the remainder of the log
> line
> that I do not care about


If the logfile has fixed width fields, you may ask the user a column range
to extract the date/time information.

--
Gabriel Genellina

 
Reply With Quote
 
 
 
 
andrew cooke
Guest
Posts: n/a
 
      02-03-2009
> > ValueError: unconverted data remains: *this is the remainder of the log *
> > line
> > that I do not care about


you could catch the ValueError and split at the ':' in the .args
attribute to find the extra data. you could then find the extra data
in the original string, use the index to remove it, and re-parse the
time.

ugly, but should work.
andrew
 
Reply With Quote
 
 
 
 
Simon Mullis
Guest
Posts: n/a
 
      11-12-2010
This was a long time ago.... But just in case anyone googling ever has
the same question, this is what I did (last year). The user just needs
to supply a strftime formatted string, such as "%A, %e %b %h:%M" and
this Class figures out the regex to use on the log entries...

class RegexBuilder(object):
"""This class is used to create the regex from the strftime string.
So, we pass it a strftime string and it returns a regex with capture
groups."""

lookup_table = { '%a' : r"(\w{3})", # locale's abbrev day name
'%A' : r"(\w{6,8})", # locale's full day name
'%b' : r"(\w{3})", # abbrev month name
'%B' : r"(\w{4,9})", # full month name
'%d' : r"(3[0-1]|[1-2]\d|0[1-9]|[1-9]|[1-9])",
# day of month
'%e' : r"([1-9]|[1-3][0-9])", # day of month, no leader
'%H' : r"(2[0-3]|[0-1]\d|\d)", # Hour (24h clock)
'%I' : r"(1[0-2]|0[1-9]|[1-9])", # Hour (12h clock)
'%j' : r"(36[0-6]|3[0-5]\d|[1-2]\d\d|0[1-9]\d|00[1-9]\
|[1-9]\d|0[1-9]|[1-9])", # Day of year
'%m' : r"(1[0-2]|0[1-9]|[1-9])", # Month as decimal
'%M' : r"([0-5]\d|\d)", # Minute
'%S' : r"(6[0-1]|[0-5]\d|\d)", # Second
'%U' : r"(5[0-3]|[0-4]\d|\d)", # Week of year (Sun = 0)
'%w' : r"([0-6])", # Weekday (Sun = 0)
'%W' : r"(5[0-3]|[0-5]\d|\d)", # Week of year (Mon = 0)
'%y' : r"(\d{2})", # Year (no century)
'%Y' : r"(\d{4})", # Year with 4 digits
'%p' : r"(AM|PM)",
'%P' : r"(am|pm)",
'%f' : r"(\d+)", # TODO: microseconds. Only in Py 2.6+
}

# Format of the keys in the table above
strftime_re = r'%\w'

def __init__(self, date_format):
r = re.compile(RegexBuilder.strftime_re)
self.created_re = r.sub(self._lookup, date_format)

def _lookup(self, match):
""" Regex lookup..."""
return RegexBuilder.lookup_table[match.group()]


> 2009/2/3 andrew cooke <(E-Mail Removed)>
>>
>> > > ValueError: unconverted data remains: *this is the remainder of the
>> > > log
>> > > line
>> > > that I do not care about

>>
>> you could catch the ValueError and split at the ':' in the .args
>> attribute to find the extra data. *you could then find the extra data
>> in the original string, use the index to remove it, and re-parse the
>> time.
>>
>> ugly, but should work.
>> andrew
>> --
>> http://mail.python.org/mailman/listinfo/python-list

>
>
>
> --
> Simon Mullis
> _________________
> http://www.velocityreviews.com/forums/(E-Mail Removed)
>

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
time.strftime('%m-%d-%Y %H:%m:%S') to log is out of order davidj411 Python 11 07-28-2009 08:24 PM
Allowing entry of a Carriage Return during data entry Mike Owen ASP .Net Web Controls 3 07-27-2006 02:34 PM
ASP Problem: "IIS log failed to write entry" in Event Log cherryparadise001@gmail.com ASP General 0 05-26-2006 01:52 AM
Form field entry directs to diff URLs based on entry? AtomicBob HTML 14 05-02-2006 07:07 AM
Form entry to Time part of database Date entry? Noozer Javascript 2 08-01-2005 08:10 PM



Advertisments