Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   python thread scheduler? (http://www.velocityreviews.com/forums/t329662-python-thread-scheduler.html)

project2501 04-07-2004 10:19 AM

python thread scheduler?
 
i'm doing some benchmarking and python is certainly fast enough (the
time.time resolution is more than good enough).

however, i am using the threading module to implement worker trheads to
hit a server with varying levels of workload.

hwoever, my result graphs (of server response time) are very flat and
these flat graphs are higher with the number of threads.

this suggests to me that the barrier i'm hitting is the python thread
schedular, not the server software being tested.

i've verified this with 2 different machines, each runnning the python
client, to hit a server and the graphs are still flat.

any ideas about how i can verify my suspicions and hwo to overcome them?
is there a switch in python that will allow me to have "very independent"
threads? forking takes too much memory and the clients machines get bogged
down beofre the server software under test.



project2501 04-07-2004 10:27 AM

Re: python thread scheduler?
 

to clarifgy, i'm seeing responec times as low as 0.3 seconds when the
client has 5 worker threads... rising to an average of about 8 seconds
with 50 threads... and more ith 100 threads.

here is what the python threads do (simplified):
while (work units remaining):
take time
send request to server
get responce from server
take time
record time difference
sleep for a small interval (to yield, and to vary the request rate)
reduce work unit by 1

i get flat graphs of responce time against request rate (which
doubles a specified number of times, after 10 work units.

i would expect the graphs to start to vary non-linearly ... especially at
higher numbers of threads (say 50 or more) and request rates of 256 per
second or more ...

Aahz 04-09-2004 05:35 PM

Re: python thread scheduler?
 
In article <pan.2004.04.07.10.27.57.297000@project2501.cor> ,
project2501 <project2501@project2501.cor> wrote:
>
>to clarifgy, i'm seeing responec times as low as 0.3 seconds when the
>client has 5 worker threads... rising to an average of about 8 seconds
>with 50 threads... and more ith 100 threads.
>
>here is what the python threads do (simplified):
> while (work units remaining):
> take time
> send request to server
> get responce from server
> take time
> record time difference
> sleep for a small interval (to yield, and to vary the request rate)
> reduce work unit by 1


Since this is almost entirely I/O bound, it *should* scale well with
threads. Question: is your server on the same machine as the client?
--
Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


All times are GMT. The time now is 09:24 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.