Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Performance of int/long in Python 3

Reply
Thread Tools

Performance of int/long in Python 3

 
 
rusi
Guest
Posts: n/a
 
      04-03-2013
On Apr 3, 9:03*am, Neil Hodgson <(E-Mail Removed)> wrote:
> rusi wrote:
> > ...
> >> a 'micro-benchmark' - I'd just like to avoid adding email access to get
> >> this over the threshold.

>
> > What does that last statement mean?

>
> * * Its a reference to a comment by Jamie Zawinski (relatively famous
> developer of Netscape Navigator and other things):


And Xemacs (which is famous in the free sw world for other things!)

>
> * * "Every program attempts to expand until it can read mail. Those
> programs which cannot so expand are replaced by ones which can."


Ok got it

 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-03-2013
On Wed, 03 Apr 2013 14:31:03 +1100, Neil Hodgson wrote:

> Sorting a million string list (all the file paths on a particular
> computer) went from 0.4 seconds with Python 3.2 to 0.78 with 3.3 so
> we're out of the 'not noticeable by humans' range. Perhaps this is still
> a 'micro-benchmark' - I'd just like to avoid adding email access to get
> this over the threshold.


I cannot confirm this performance regression. On my laptop (Debian Linux,
not Windows), I can sort a million file names in approximately 1.2
seconds in both Python 3.2 and 3.3. There is no meaningful difference in
speed between the two versions.



--
Steven
 
Reply With Quote
 
 
 
 
Terry Jan Reedy
Guest
Posts: n/a
 
      04-03-2013
On 4/3/2013 1:32 AM, Steven D'Aprano wrote:
> On Wed, 03 Apr 2013 14:31:03 +1100, Neil Hodgson wrote:
>
>> Sorting a million string list (all the file paths on a particular
>> computer) went from 0.4 seconds with Python 3.2 to 0.78 with 3.3 so
>> we're out of the 'not noticeable by humans' range. Perhaps this is still
>> a 'micro-benchmark' - I'd just like to avoid adding email access to get
>> this over the threshold.


What system *and* what compiler and compiler options. Unless 3.2 and 3.3
are both compiler with the same compiler and settings, we do not know
the source of the difference.

> I cannot confirm this performance regression. On my laptop (Debian Linux,
> not Windows), I can sort a million file names in approximately 1.2
> seconds in both Python 3.2 and 3.3. There is no meaningful difference in
> speed between the two versions.


I am guessing that Neil's undisclosed system (that I can see) is
Windows, since other benchmarks have been more different on Windows than
on *nix. Given that we *know* that the 3.2 and 3.3 distribution are
compiled with different compilers and run with different C runtimes, it
is possible that some of the difference is from that and not from python
at all.

tjr



 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      04-03-2013
On Wed, Apr 3, 2013 at 3:03 PM, Neil Hodgson <(E-Mail Removed)> wrote:
> rusi wrote:
> "Every program attempts to expand until it can read mail. Those programs
> which cannot so expand are replaced by ones which can."


In my personal experience, it's calculators. I put command-line
calculators into *everything*... often in the form of more general
executors, and thus restricted to admins, but it's still a calculator.

For some reason, the ability to type "calc 1+2" and get back 3 is very
satisfying to me. You know, in case I ever forget what one plus two
makes.

ChrisA
 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      04-03-2013
On Wed, Apr 3, 2013 at 4:32 PM, Steven D'Aprano
<(E-Mail Removed)> wrote:
> On Wed, 03 Apr 2013 14:31:03 +1100, Neil Hodgson wrote:
>
>> Sorting a million string list (all the file paths on a particular
>> computer) went from 0.4 seconds with Python 3.2 to 0.78 with 3.3 so
>> we're out of the 'not noticeable by humans' range. Perhaps this is still
>> a 'micro-benchmark' - I'd just like to avoid adding email access to get
>> this over the threshold.

>
> I cannot confirm this performance regression. On my laptop (Debian Linux,
> not Windows), I can sort a million file names in approximately 1.2
> seconds in both Python 3.2 and 3.3. There is no meaningful difference in
> speed between the two versions.


I'd be curious to know the sorts of characters used. Given that it's
probably a narrow-vs-wide Python difference we're talking here, the
actual distribution of codepoints may well make a difference.

ChrisA
 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      04-03-2013
On Wed, Apr 3, 2013 at 5:29 PM, Neil Hodgson <(E-Mail Removed)> wrote:
> Chris Angelico:
>
>
>> I'd be curious to know the sorts of characters used. Given that it's
>> probably a narrow-vs-wide Python difference we're talking here, the
>> actual distribution of codepoints may well make a difference.

>
>
> I was going to upload it but then I thought of potential client
> -confidentiality problems and the need to audit a list that long.


Hmm. I was about to say "Can you just do a quick collections.Counter()
of the string widths in 3.3, as an easy way of seeing which ones use
BMP or higher characters", but I can't find a simple way to query a
string's width. Can't see it as a method of the string object, nor in
the string or sys modules. It ought to be easy enough at the C level -
just look up the two bits representing 'kind' - but I've not found it
exposed to Python. Is there anything?

ChrisA
 
Reply With Quote
 
Ian Kelly
Guest
Posts: n/a
 
      04-03-2013
On Wed, Apr 3, 2013 at 12:52 AM, Chris Angelico <(E-Mail Removed)> wrote:
> Hmm. I was about to say "Can you just do a quick collections.Counter()
> of the string widths in 3.3, as an easy way of seeing which ones use
> BMP or higher characters", but I can't find a simple way to query a
> string's width. Can't see it as a method of the string object, nor in
> the string or sys modules. It ought to be easy enough at the C level -
> just look up the two bits representing 'kind' - but I've not found it
> exposed to Python. Is there anything?


4 if max(map(ord, s)) > 0xffff else 2 if max(map(ord, s)) > 0xff else 1
 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      04-03-2013
On Wed, Apr 3, 2013 at 6:06 PM, Ian Kelly <(E-Mail Removed)> wrote:
> On Wed, Apr 3, 2013 at 12:52 AM, Chris Angelico <(E-Mail Removed)> wrote:
>> Hmm. I was about to say "Can you just do a quick collections.Counter()
>> of the string widths in 3.3, as an easy way of seeing which ones use
>> BMP or higher characters", but I can't find a simple way to query a
>> string's width. Can't see it as a method of the string object, nor in
>> the string or sys modules. It ought to be easy enough at the C level -
>> just look up the two bits representing 'kind' - but I've not found it
>> exposed to Python. Is there anything?

>
> 4 if max(map(ord, s)) > 0xffff else 2 if max(map(ord, s)) > 0xff else 1


Yeah, that's iterating over the whole string (twice, if it isn't width
4). The system already knows what the size is, I was hoping for an
uber-quick inspection of the string header.

ChrisA
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-03-2013
On Wed, 03 Apr 2013 18:24:25 +1100, Chris Angelico wrote:

> On Wed, Apr 3, 2013 at 6:06 PM, Ian Kelly <(E-Mail Removed)> wrote:
>> On Wed, Apr 3, 2013 at 12:52 AM, Chris Angelico <(E-Mail Removed)>
>> wrote:
>>> Hmm. I was about to say "Can you just do a quick collections.Counter()
>>> of the string widths in 3.3, as an easy way of seeing which ones use
>>> BMP or higher characters", but I can't find a simple way to query a
>>> string's width. Can't see it as a method of the string object, nor in
>>> the string or sys modules. It ought to be easy enough at the C level -
>>> just look up the two bits representing 'kind' - but I've not found it
>>> exposed to Python. Is there anything?

>>
>> 4 if max(map(ord, s)) > 0xffff else 2 if max(map(ord, s)) > 0xff else 1

>
> Yeah, that's iterating over the whole string (twice, if it isn't width
> 4).


Then don't write it as a one-liner

n = max(map(ord, s))
4 if n > 0xffff else 2 if n > 0xff else 1


Here's another way:


(sys.getsizeof(s) - sys.getsizeof(''))/len(s)

should work.


There's probably also a way to do it using ctypes.



> The system already knows what the size is, I was hoping for an
> uber-quick inspection of the string header.


I'm not sure that I would want strings to have a method reporting this,
but it might be nice to have a function in the inspect module to do so.



--
Steven
 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      04-03-2013
On Wed, Apr 3, 2013 at 6:53 PM, Steven D'Aprano
<(E-Mail Removed)> wrote:
> Here's another way:
>
>
> (sys.getsizeof(s) - sys.getsizeof(''))/len(s)
>
> should work.


Hmm, I had been under the impression that there was a certain "base
length" below which strings all had the same size. Yes, that also
works; though again, it's something that can be directly queried, at
the C level.

> There's probably also a way to do it using ctypes.
>
>> The system already knows what the size is, I was hoping for an
>> uber-quick inspection of the string header.

>
> I'm not sure that I would want strings to have a method reporting this,
> but it might be nice to have a function in the inspect module to do so.


Yeah, that's why I also looked in 'sys'; 'inspect' might well be a
good place for it, too. But it seems such a function doesn't exist,
which is what I was asking.

ChrisA
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Performance Tutorials Services - Boosting Performance by DisablingUnnecessary Services on Windows XP Home Edition Software Engineer Javascript 0 06-10-2011 02:18 AM
Re: Performance (pystone) of python 2.4 lower then python 2.3 ??? Andreas Kostyrka Python 0 12-17-2004 02:00 PM
Performance (pystone) of python 2.4 lower then python 2.3 ??? Lucas Hofman Python 13 12-16-2004 03:24 AM
RE: Straw poll on Python performance (was Re: Python is far from atop performer ...) Robert Brewer Python 1 01-10-2004 06:54 AM
Web Form Performance Versus Single File Performance jm ASP .Net 1 12-12-2003 11:14 PM



Advertisments