Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Function application optimization.

Reply
Thread Tools

Function application optimization.

 
 
Jacek Generowicz
Guest
Posts: n/a
 
      12-12-2003
Given

fncs = [func1, func2, ..., funcN]
args = [arg1, arg2, ..., argN]

How should one spell

results = map(lambda f,a: f(a), fncs, args)

in order to get the result most quickly ?



Unfortunately "apply" takes a tuple of arguments, and there is no
"funcall"[*] in Python.

[*] def funcall(fn, *args):
return fn(*args)

 
Reply With Quote
 
 
 
 
Bengt Richter
Guest
Posts: n/a
 
      12-12-2003
On 12 Dec 2003 10:47:50 +0100, Jacek Generowicz <(E-Mail Removed)> wrote:

>Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
>How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
>in order to get the result most quickly ?
>
>
>
>Unfortunately "apply" takes a tuple of arguments, and there is no
>"funcall"[*] in Python.
>
>
>[*] def funcall(fn, *args):
> return fn(*args)
>


>>> fncs = [lambda x,f='func_%s(%%s)'%i:f%x for i in xrange(4)]
>>> args = 'zero one two three'.split()
>>> map(lambda f,a: f(a), fncs, args)

['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']

I'd probably try a list comprehension

>>> [f(a) for f,a in zip(fncs,args)]

['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']

Regards,
Bengt Richter
 
Reply With Quote
 
 
 
 
Duncan Booth
Guest
Posts: n/a
 
      12-12-2003
Jacek Generowicz <(E-Mail Removed)> wrote in
news:(E-Mail Removed):

> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>

Well, the way you wrote it isn't actually bad. The obvious alternative
(using a list comprehension) is slightly slower, but there isn't an awful
lot to choos between any of them, I suspect most of the time goes in the
actual f(a) function call. Any of these methods runs at about 300,000
function calls/second on my slow laptop.

>>> setup = """import itertools

def fn1(a): pass
fns = [fn1] * 100
args = [0] * 100
"""
>>> stmt1 = "result = [ f(a) for (f,a) in itertools.izip(fns, args) ]"
>>> stmt2 = "result = [ f(a) for (f,a) in zip(fns, args) ]"
>>> t = timeit.Timer(stmt1, setup)
>>> t.repeat(3,10000)

[3.5571303056673855, 3.5537404893639177, 3.5594278043718077]
>>> t = timeit.Timer(stmt2, setup)
>>> t.repeat(3,10000)

[3.893281967400867, 3.87834794645687, 3.8829105375124868]
>>> setup = """import itertools

def fn1(a): pass
fns = [fn1] * 1000
args = [0] * 1000
"""
>>> t = timeit.Timer(stmt1, setup)
>>> t.repeat(3,1000)

[3.3503928571927304, 3.3343195853104248, 3.3495254285111287]
>>> t = timeit.Timer(stmt2, setup)
>>> t.repeat(3,1000)

[3.8062683944467608, 3.7946001516952492, 3.7881063096007779]
>>> stmt3 = "results = map(lambda f,a: f(a), fns, args)"
>>> t = timeit.Timer(stmt3, setup)
>>> t.repeat(3,1000)

[3.3275902384241363, 3.3010907810909202, 3.3174872784110789]

The f(a) call is taking about half the time in any of these methods, so you
aren't going to get very much improvement whatever you do to the loop.

--
Duncan Booth http://www.velocityreviews.com/forums/(E-Mail Removed)
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
 
Reply With Quote
 
Bruno Desthuilliers
Guest
Posts: n/a
 
      12-12-2003
Jacek Generowicz wrote:
> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>


I dont know if it will be faster (timeit might tell you this), but what
about something as simple stupid as:

results = []
nbfuns = len(fncs)
for i in range(len):
results.append(fncs[i](args[i]))

Note that you must have at least len(fncs) args.


HTH,
Bruno

 
Reply With Quote
 
Jacek Generowicz
Guest
Posts: n/a
 
      12-12-2003
(E-Mail Removed) (Bengt Richter) writes:

> I'd probably try a list comprehension
>
> >>> [f(a) for f,a in zip(fncs,args)]

> ['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']


Yup, been there, done that, it's slower. (I guess I should have
mentioned that in the original post.)
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      12-12-2003
Jacek Generowicz wrote:

> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>


If you can afford to destroy the args list, a tiny speed gain might be in
for you (not verified):

for index, func in enumerate(fncs):
args[index] = func[index]

Peter
 
Reply With Quote
 
John Roth
Guest
Posts: n/a
 
      12-12-2003

"Jacek Generowicz" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>
>
>
> Unfortunately "apply" takes a tuple of arguments, and there is no
> "funcall"[*] in Python.
>
>
>[*] def funcall(fn, *args):
> return fn(*args)



Building on a couple of other responses:

Untested code:

fncs = [func1, func2, ..., funcN]
args = [arg1, arg2, ..., argN]
results = []
for function, arguement in zip(fncs, args):
results.append(function(arguement))

Notice the use of zip() to put the two lists together.
I haven't timed it, but removing an extra layer of
function call has got to be faster. Likewise, zip
and tuple unpacking is most likely going to be
faster than indexing every time through the loop.
The append, on the other hand, might slow things
down a bit.

John Roth

>



 
Reply With Quote
 
Duncan Booth
Guest
Posts: n/a
 
      12-12-2003
"John Roth" <(E-Mail Removed)> wrote in
news:(E-Mail Removed):

> Building on a couple of other responses:
>
> Untested code:
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
> results = []
> for function, arguement in zip(fncs, args):
> results.append(function(arguement))
>
> Notice the use of zip() to put the two lists together.
> I haven't timed it, but removing an extra layer of
> function call has got to be faster. Likewise, zip
> and tuple unpacking is most likely going to be
> faster than indexing every time through the loop.
> The append, on the other hand, might slow things
> down a bit.


Yes, getting rid of the append does indeed speed things up. On the same
system as I posted timings for the list comprehension, the fastest way I've
found so far is to get rid of the appends by preallocating the list, and to
go for the plain old simple technique of writing a for loop out explicitly.
Using 'enumerate' to avoid the lookup on one of the input lists is nearly
as fast, but not quite.

>>> setup = """import itertools

def fn1(a): pass
fns = [fn1] * 1000
args = [0] * 1000
"""

>>> stmt1 = """result = args[:]

for i in range(len(fns)):
result[i] = fns[i](args[i])
"""
>>> min(timeit.Timer(stmt1, setup).repeat(3,1000))

2.9747384094916924
>>> stmt3 = "results = map(lambda f,a: f(a), fns, args)"
>>> min(timeit.Timer(stmt3, setup).repeat(3,1000))

3.3257092731055309
>>>


--
Duncan Booth (E-Mail Removed)
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      12-12-2003
Peter Otten wrote:

> for index, func in enumerate(fncs):
> args[index] = func[index]


Oops,
args[index] = func(args[index])

And it's much slower than result.append(func(args[index]), too

Peter

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Partial Function Application -- Advantages over normal function? Dave Angel Python 8 07-19-2011 05:15 PM
Partial Function Application -- Advantages over normal function? Kurian Thayil Python 3 07-18-2011 08:17 PM
How override ALL function calls? (Is there a "function call function"?) seberino@spawar.navy.mil Python 2 08-01-2005 12:38 PM
write a function such that when ever i call this function in some other function .it should give me tha data type and value of calling function parameter komal C++ 6 01-25-2005 11:13 AM
Passing a C++ object's member function to a C function expecing a function pointer! James Vanns C++ 7 01-21-2004 02:39 AM



Advertisments