Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > SCons build tool speed

Reply
Thread Tools

SCons build tool speed

 
 
ted
Guest
Posts: n/a
 
      02-12-2005
How does the speed of the Scons build tool compare with Ant? Right now with
out Ant builds take around an hour. Hoping to speed that up.

TIA,
Ted


 
Reply With Quote
 
 
 
 
Christopher De Vries
Guest
Posts: n/a
 
      02-13-2005
On Sat, Feb 12, 2005 at 07:16:02PM +0000, ted wrote:
> How does the speed of the Scons build tool compare with Ant?


I would recommend asking this question on http://www.velocityreviews.com/forums/(E-Mail Removed) , but my
impressions is that most of the time is probably spent in the compiler. If you
are working on a java project you could try switching from javac to jikes and
that might improve your time, though it has been a while since I used jikes.

Chris
 
Reply With Quote
 
 
 
 
Peter Hansen
Guest
Posts: n/a
 
      02-13-2005
ted wrote:
> How does the speed of the Scons build tool compare with Ant? Right now with
> out Ant builds take around an hour. Hoping to speed that up.


Don't tools like Scons, Ant, and for that matter "make" just
execute other programs? So that 99% of the time is consumed
external to the Scons, Ant, or make process itself? Why
do you think the speed of the build tool is of any significance
compared to the time for things like compilers and linkers
to execute?

-Peter
 
Reply With Quote
 
Mike Meyer
Guest
Posts: n/a
 
      02-13-2005
Peter Hansen <(E-Mail Removed)> writes:

> ted wrote:
>> How does the speed of the Scons build tool compare with Ant? Right
>> now with out Ant builds take around an hour. Hoping to speed that up.

>
> Don't tools like Scons, Ant, and for that matter "make" just
> execute other programs? So that 99% of the time is consumed
> external to the Scons, Ant, or make process itself? Why
> do you think the speed of the build tool is of any significance
> compared to the time for things like compilers and linkers
> to execute?


Actually, these tools do a number of things oter than run other
programs. For instance, they parse input files to construct a build
tree, walk the build tree figuring out what needs to be rebuilt, and
only *then* do they get around to executing those other
programs. Assuming that the work prior to running those other programs
is "neglible" can lead to some truly atrocious build-time behavior. In
fact, one major advantage that ant/scons/jam/etc. have over Make is
that they avoid the defacto standard "recursive make" build
system. See <URL:
http://www.pcug.org.au/~millerp/rmch...cons-harm.html > for
details on that.

<mike
--
Mike Meyer <(E-Mail Removed)> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
 
Reply With Quote
 
Neil Hodgson
Guest
Posts: n/a
 
      02-13-2005
ted:

> How does the speed of the Scons build tool compare with Ant?
> Right now with out Ant builds take around an hour. Hoping to
> speed that up.


Scons emphasises accuracy over speed and is normally a little slower than
other build tools although still fast enough for most purposes. One cause of
slowness is that it reads the source files, traces include files and
checksums them all rather than relying on file times. There are some things
you can do to speed it up:
http://www.scons.org/cgi-bin/wiki/GoFastButton

Neil


 
Reply With Quote
 
PA
Guest
Posts: n/a
 
      02-13-2005

On Feb 13, 2005, at 12:20, Neil Hodgson wrote:

> http://www.scons.org/cgi-bin/wiki/GoFastButton


Out of curiosity, why does scons uses MD5 by default? Is that not, er,
somewhat heavy handed for most practical purpose?

Would not some sort of lightweight CRC be "good enough" to start with?

http://samba.anu.edu.au/rsync/tech_report/node3.html

Cheers

--
PA, Onnay Equitursay
http://alt.textdrive.com/

 
Reply With Quote
 
knight@baldmt.com
Guest
Posts: n/a
 
      02-13-2005
Hi Neil, ted, et al.--

> > How does the speed of the Scons build tool compare with Ant?
> > Right now with out Ant builds take around an hour. Hoping to
> > speed that up.

>
> Scons emphasises accuracy over speed and is normally a little
> slower than other build tools although still fast enough for most
> purposes. One cause of slowness is that it reads the source files,
> traces include files and checksums them all rather than relying
> on file times. There are some things you can do to speed it up:
> http://www.scons.org/cgi-bin/wiki/GoFastButton


In anticipation of the next release of SCons, we've been doing
a *lot* of work on profiling the performance and eliminating
bottlenecks.

It turns out that scanning the source files and performing the MD5
calculation is not the dominant factor that we've all assumed it is.
There were some other inefficiencies in some of our algorithms that
were much more significant, including some unnecessary disk scans,
recomputing the same dependencies for every target in a list of
targets generated by one command, and repeated just-in-time creation
of some internal objects that could be created just once and cached.

That said, SCons will never be as fast as Make, because it *is*
doing more for you under the covers. Although the next version
won't necessarily speed up every configuration (the bottlenecks
are extremely configuration dependent), it should be a significant
improvement for many configurations out there.

--SK

 
Reply With Quote
 
knight@baldmt.com
Guest
Posts: n/a
 
      02-13-2005
Hi Peter--

> > How does the speed of the Scons build tool compare with
> > Ant? Right now with out Ant builds take around an hour. Hoping
> > to speed that up.

>
> Don't tools like Scons, Ant, and for that matter "make" just
> execute other programs? So that 99% of the time is consumed
> external to the Scons, Ant, or make process itself? Why
> do you think the speed of the build tool is of any significance
> compared to the time for things like compilers and linkers
> to execute?


You're right that build times are dominated by the external commands
when rebuilds occur. If nothing needs to be rebuilt, though, the
wall-clock time is obviously dominated by how long it takes for
the build tool to decide that. A build tool that doesn't decide
whether or not targets are up-to-date *and* dispatch the commnds
to rebuild them reasonably quickly and efficiently doesn't do a
good job of serving its users.

--SK

 
Reply With Quote
 
Peter Hansen
Guest
Posts: n/a
 
      02-14-2005
(E-Mail Removed) wrote:
> You're right that build times are dominated by the external commands
> when rebuilds occur. If nothing needs to be rebuilt, though, the
> wall-clock time is obviously dominated by how long it takes for
> the build tool to decide that. A build tool that doesn't decide
> whether or not targets are up-to-date *and* dispatch the commnds
> to rebuild them reasonably quickly and efficiently doesn't do a
> good job of serving its users.


That answer, combined with Mike's response pointing out
that tools more sophisticated than basic "make" actually
can delve into the source and identify the dependencies,
removes my confusion. Thanks.

-Peter
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
build flow? SCons? AAP? process creation? corey.coughlin@comcast.net Python 3 04-19-2005 10:47 PM
SWsoft Acronis Disk Director Suite 9.0 Build 508, Acronis OS Selector 8.0 Build 917, Acronis Partition Expert 2003 Build 292, Acronis Power Utilities 2004 Build 502, F-SECURE.ANTI vIRUS.PROXY v1.10.17.WINALL, F-SECURE.ANTI vIRUS v5.50.10260 for CITRI vvcd Computer Support 0 09-25-2004 01:38 AM
Rake vs SCons vs Boost.Build v2 vs Perforce Jam Ruby Script Ruby 0 08-21-2004 05:56 PM
speed speed speed a.metselaar Computer Support 14 12-30-2003 03:34 AM
java tool to test disk i/o, processor speed, and network speed efiedler Java 1 10-09-2003 03:36 PM



Advertisments