Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Pyparsing performance boost using Python 2.6b1

Reply
Thread Tools

Pyparsing performance boost using Python 2.6b1

 
 
Paul McGuire
Guest
Posts: n/a
 
      06-19-2008
I just ran my pyparsing unit tests with the latest Python 2.6b1
(labeled internally as Python 2.6a3 - ???), and the current 1.5.0
version of pyparsing runs with no warnings or regressions.

I was pleasantly surprised by the improved performance. The most
complex parser I have is the Verilog parser, and I have just under 300
sample input files, so the test gets a chance to run over a range of
code and source inputs. Here are the lines/second parsing for the
Verilog data (higher numbers are better):

Python V2.5.1 Python V2.6b1
base
209.2 307.0

with packrat optimization enabled
349.8 408.0

This is a huge percentage improvement, anywhere from 15-50%! I do not
know what it is about 2.6 that runs so much faster, but given that
packratting achieves somewhat less improvement, I would guess that the
2.6 performance comes from some optimization in making function calls,
or in GC of local variables when functions are completed (since
packratting is a form of internal memoization of parse expressions,
which thereby avoids duplicate calls to parsing functions).

Using psyco typically gives another 30-50% performance improvement,
but there is no psyco available for 2.6 yet, so I skipped those tests
for now.

-- Paul
 
Reply With Quote
 
 
 
 
dwahli@gmail.com
Guest
Posts: n/a
 
      06-20-2008
On Jun 19, 12:40*pm, Paul McGuire <(E-Mail Removed)> wrote:
> I just ran my pyparsing unit tests with the latest Python 2.6b1
> (labeled internally as Python 2.6a3 - ???), and the current 1.5.0
> version of pyparsing runs with no warnings or regressions.
>
> I was pleasantly surprised by the improved performance. *The most
> complex parser I have is the Verilog parser, and I have just under 300
> sample input files, so the test gets a chance to run over a range of
> code and source inputs. *Here are the lines/second parsing for the
> Verilog data (higher numbers are better):
>
> *Python V2.5.1 * Python V2.6b1
> base
> *209.2 * * * * * * * *307.0
>
> with packrat optimization enabled
> *349.8 * * * * * * * *408.0
>
> This is a huge percentage improvement, anywhere from 15-50%! *I do not
> know what it is about 2.6 that runs so much faster, but given that
> packratting achieves somewhat less improvement, I would guess that the
> 2.6 performance comes from some optimization in making function calls,
> or in GC of local variables when functions are completed (since
> packratting is a form of internal memoization of parse expressions,
> which thereby avoids duplicate calls to parsing functions).
>
> Using psyco typically gives another 30-50% performance improvement,
> but there is no psyco available for 2.6 yet, so I skipped those tests
> for now.
>
> -- Paul


Tests was made on Windows ?
Just try pybench on 2.6 and 2.5, and 2.6 is 17% better than 2.5 !
Very nice !

But I suspect the new compiler with Profile Guided Optimization (PGO)
to be the main cause of this nice boost...
 
Reply With Quote
 
 
 
 
John Machin
Guest
Posts: n/a
 
      06-20-2008
On Jun 19, 8:40 pm, Paul McGuire <(E-Mail Removed)> wrote:
> I just ran my pyparsing unit tests with the latest Python 2.6b1
> (labeled internally as Python 2.6a3 - ???),


Hi, Paul. If it says 2.6a3, that's what it is. Look at the thread of
replies to Barry Warsaw's announcement of 2.6b1 ... [from memory]
there was a delay expected before MvL would make a Windows msi
available, and in the meantime the download page would point to "the
alpha version".

Cheers,
John
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Boost::any and boost::lambda with std::find_if Misiu C++ 3 01-31-2007 05:46 PM
#include <boost/shared_ptr.hpp> or #include "boost/shared_ptr.hpp"? Colin Caughie C++ 1 08-29-2006 02:19 PM
Problems mixing boost::lambda::bind and boost::shared_ptr.. Toby Bradshaw C++ 6 06-02-2006 04:12 PM
Any Boost Experts out there for Boost.Regex? Richard Latter C++ 2 05-17-2004 03:12 PM
Boost + Python C/API: Mixing python return types with boost return types Steve Knight Python 2 10-10-2003 10:11 AM



Advertisments