Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Tips for load balancing multiple Python apps on dual/quad coreprocessors?

Reply
Thread Tools

Tips for load balancing multiple Python apps on dual/quad coreprocessors?

 
 
Malcolm Greene
Guest
Posts: n/a
 
      04-07-2008
I'm looking for tips on how to load balance running multiple Python
applications in multi-CPU environments. My understanding is that Python
applications and their threads are limited to a specific CPU.

Background: I have a Python utility that processes email messages. I
suspect there's a lot of idle time while this utility waits on a remote
email server. I would like to run as many simultaneous copies of this
utility as possible without overwhelming the server these utilities are
running on. My thought is that I should write a dispatcher that monitors
CPU load and launches/cancels multiple instances of my utility with
specific job queues to process.

Is there a cross-platform way to monitor CPU load?

Is there a pre-existing Python module I should be looking at for
building (subclassing) my dispatcher?

Thanks!
Malcolm

 
Reply With Quote
 
 
 
 
Paul Rubin
Guest
Posts: n/a
 
      04-07-2008
"Malcolm Greene" <(E-Mail Removed)> writes:
> Is there a cross-platform way to monitor CPU load?


Cross-platform: not that I know of.

Linux: /proc/loadav (load average),
/proc/cpuinfo (to figure out number of cpu's).

You want the load average and # of cpu's to be about equal,
i.e. all cpu's should be kept busy but not overloaded.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Diff between load balancing & load sharing palas_123 Cisco 1 12-28-2009 11:32 PM
Load Balancing Multiple Cisco Routers with Multiple ADSL circuits Big Phil Cisco 3 05-01-2007 04:22 PM
Load balancing multiple bgp full feeds + some backups pshemko Cisco 1 12-08-2006 10:47 PM
Running python apps from within python apps aph Python 4 01-14-2006 04:06 PM
Load Balancing / Load Sharing over parallel paths ciscortp@hotmail.com Cisco 1 11-21-2005 03:57 PM



Advertisments