Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Compile time evaluation of dictionaries

Reply
Thread Tools

Re: Compile time evaluation of dictionaries

 
 
Terry Reedy
Guest
Posts: n/a
 
      03-10-2011
On 3/10/2011 11:23 AM, Gerald Britton wrote:
> Today I noticed that an expression like this:
>
> "one:%(one)s two:%(two)s" % {"one": "is the loneliest number", "two":
> "can be as bad as one"}
>
> could be evaluated at compile time, but is not:


In fact, it could be evaluated at writing time .
This would be an example of constant folding, except that a dict is not,
in itself, a constant.

>>>> dis(compile(

> ... '"one:%(one)s two:%(two)s" % {"one": "is the loneliest number",
> "two": "can be as bad as one"}',
> ... '','exec'))
> 1 0 LOAD_CONST 0 ('one:%(one)s two:%(two)s')
> 3 BUILD_MAP 2
> 6 LOAD_CONST 1 ('is the loneliest number')
> 9 LOAD_CONST 2 ('one')
> 12 STORE_MAP
> 13 LOAD_CONST 3 ('can be as bad as one')
> 16 LOAD_CONST 4 ('two')
> 19 STORE_MAP
> 20 BINARY_MODULO
> 21 POP_TOP
> 22 LOAD_CONST 5 (None)
> 25 RETURN_VALUE
>>>>

>
> Any idea why Python works this way?


Because that is how the language is defined.

> I see that, in 3.2, an
> optimization was done for sets (See "Optimizations" at
> http://docs.python.org/py3k/whatsnew/3.2.html) though I do not see
> anything similar for dictionaries.


I presume you mean this:

"Pythonís peephole optimizer now recognizes patterns such x in {1, 2, 3}
as being a test for membership in a set of constants. The optimizer
recasts the set as a frozenset and stores the pre-built constant.

Now that the speed penalty is gone, it is practical to start writing
membership tests using set-notation. This style is both semantically
clear and operationally fast:

extension = name.rpartition('.')[2]
if extension in {'xml', 'html', 'xhtml', 'css'}:
handle(name)

(Patch and additional tests contributed by Dave Malcolm; issue 6690)."

This is possible because: sets have a frozen counterpart, and cpython
*sometimes* follows an 'as if' rule for optimizations. There is no
frozendict, so the same pattern could not be followed.

Note first that the optimization is *not* an example of constant
folding, but recognition that an unbound mutable cannot be mutated, and
therefore, if all its contents are fixed, can be 'cast' to constant form
at compile time.

Note second that the optimization does not apply to all such set
instances, but only to this particular 'in' context. It follows a
similar optimization for turning lists into tuples (although in this
latter case, the programmer could have remembered to use () instead of []).

The optimization *did* happen because various people started an issue,
wrote a patch, reviewed the patch, and committed it to the cpython
source. You might read the issue if you have not:

http://bugs.python.org/issue6690

Terry Jan Reedy


 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      03-11-2011
On Thu, 10 Mar 2011 17:40:40 -0500, Terry Reedy wrote:

> On 3/10/2011 11:23 AM, Gerald Britton wrote:
>> Today I noticed that an expression like this:
>>
>> "one:%(one)s two:%(two)s" % {"one": "is the loneliest number", "two":
>> "can be as bad as one"}
>>
>> could be evaluated at compile time, but is not:

>
> In fact, it could be evaluated at writing time .


True, but why do stuff when the compiler can do it for you? Any constant
folding could be done at writing time, but readability and maintenance
dictates that we write something like:

C = 1.0/7

rather than

C = 0.14285714285714285


> This would be an
> example of constant folding, except that a dict is not, in itself, a
> constant.


Nevertheless, since the dict only exists for a single operation, it might
as well be a constant. In general, Python can't safely make many
assumptions about compile-time behaviour, since nearly anything can be
modified at run-time, but the behaviour of built-in literals is one thing
that can't change.

I don't see any reason why Python couldn't optimize the above at compile-
time, and I can only think of two reasons why it won't:

- lack of interest from anyone willing and able to write a patch;
- the added complexity may be more than the benefit gained.


--
Steven
 
Reply With Quote
 
 
 
 
Chris Rebert
Guest
Posts: n/a
 
      03-11-2011
On Thu, Mar 10, 2011 at 4:15 PM, Steven D'Aprano
<(E-Mail Removed)> wrote:
> On Thu, 10 Mar 2011 17:40:40 -0500, Terry Reedy wrote:
>> On 3/10/2011 11:23 AM, Gerald Britton wrote:
>>> Today I noticed that an expression like this:
>>>
>>> "one:%(one)s two:%(two)s" % {"one": "is the loneliest number", "two":
>>> "can be as bad as one"}
>>>
>>> could be evaluated at compile time, but is not:

>>
>> In fact, it could be evaluated at writing time .

>
> True, but why do stuff when the compiler can do it for you?

<snip>
> I don't see any reason why Python couldn't optimize the above at compile-
> time, and I can only think of two reasons why it won't:
>
> - lack of interest from anyone willing and able to write a patch;
> - the added complexity may be more than the benefit gained.


3. %-formatting is "obsolete and may go away in future versions of Python."
(See http://docs.python.org/py3k/library/...ing-operations
)

Cheers,
Chris
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      03-11-2011
On Thu, 10 Mar 2011 16:27:17 -0800, Chris Rebert wrote:


> 3. %-formatting is "obsolete and may go away in future versions of
> Python." (See
> http://docs.python.org/py3k/library/...ng-formatting-

operations
> )


There is an awful lot of opposition to that. If it ever happens, it
probably won't happen until Python4, but even if it happens sooner, you
could replace

"one:%(one)s two:%(two)s" % \
{"one": "is the loneliest number", "two": "can be as bad as one"}

with the format string equivalent, still using literals, and the same
constant-folding optimization could occur.

"one:{one} two:{two}".format(
**{"one": "is the loneliest number", "two": "can be as bad as one"})

Admittedly, it would require a significantly smarter peephole optimizer
to recognise this as a constant, which pushes up the complexity for no
additional gain, but the principle still applies.



--
Steven
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Compile time evaluation of dictionaries Gerald Britton Python 0 03-14-2011 02:21 PM
Compile time evaluation of dictionaries Gerald Britton Python 2 03-12-2011 06:57 AM
computation at compile time i.e. compile time functions usingtemplates Carter C++ 2 03-04-2009 06:43 PM
updating dictionaries from/to dictionaries Brandon Python 12 08-15-2008 12:35 AM
Pickling dictionaries containing dictionaries: failing,recursion-style! lysdexia Python 6 12-02-2007 12:03 AM



Advertisments