Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: to py or not to py ?

Reply
Thread Tools

Re: to py or not to py ?

 
 
Carl J. Van Arsdall
Guest
Posts: n/a
 
      06-28-2006
Robert Kern wrote:
>> Carl , what are the problems that could arise with threading ??
>>

Because of the GIL only one thread can actually run at a time. So if
you are going for speed (say you have an SMP box) and your two requests
require significant computation, you'd want each processor to take on a
little bit of that work to get things done faster. Now say you want
these two to be computed simultaneously and you are using python, you
won't get that, one thread will process and then the other will process
as the GIL is passed around between the python threads. Now if your
bottleneck is IO then you are in good shape for python threads (Your
machine only has one network port and you don't have a lot of
computation overhead, so your single processor can serve things up
faster than your network card can take it).

So its not problems that arise, its more like inefficiency. Anyhow, it
all depends on what you are trying to do and the needs of your
application.

-carl

--

Carl J. Van Arsdall
http://www.velocityreviews.com/forums/(E-Mail Removed)
Build and Release
MontaVista Software

 
Reply With Quote
 
 
 
 
Tom Plunket
Guest
Posts: n/a
 
      06-28-2006
Carl J. Van Arsdall wrote:

> Because of the GIL only one thread can actually run at a time.


I've recently been wondering about this, since in the work I do, a lot
of time is spent doing disk I/O. So if I want the UI to remain
responsive, I could spawn an IO thread to handle requests, and do a
pretty simple "just whack new requests onto the queue" without locks
since I'm guaranteed to not have the IO thread read at the same time
as the requestor thread?

....what exactly constitutes an atomic operation in Python, anyway?

e.g.

class IoThread:
# ...

# called from the other thread...
def RequestFile(self, name):
self.fileQueue.append(name)

# called during the IO thread
def GetNextFile(self);
next = self.fileQueue[0]
self.fileQueue.pop(0)
return next

?
-tom!
 
Reply With Quote
 
 
 
 
Steve Holden
Guest
Posts: n/a
 
      06-29-2006
Tom Plunket wrote:
> Carl J. Van Arsdall wrote:
>
>
>>Because of the GIL only one thread can actually run at a time.

>
>
> I've recently been wondering about this, since in the work I do, a lot
> of time is spent doing disk I/O. So if I want the UI to remain
> responsive, I could spawn an IO thread to handle requests, and do a
> pretty simple "just whack new requests onto the queue" without locks
> since I'm guaranteed to not have the IO thread read at the same time
> as the requestor thread?
>
> ...what exactly constitutes an atomic operation in Python, anyway?
>

Standard (and excellent) advice to those new to Python: use threading,
not thread, and have threads communicate using Queue.Queue(s).

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Love me, love my blog http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

 
Reply With Quote
 
Carl J. Van Arsdall
Guest
Posts: n/a
 
      06-29-2006
Tom Plunket wrote:
> Carl J. Van Arsdall wrote:
>
>
>> Because of the GIL only one thread can actually run at a time.
>>

>
> I've recently been wondering about this, since in the work I do, a lot
> of time is spent doing disk I/O. So if I want the UI to remain
> responsive, I could spawn an IO thread to handle requests, and do a
> pretty simple "just whack new requests onto the queue" without locks
> since I'm guaranteed to not have the IO thread read at the same time
> as the requestor thread?
>
> ...what exactly constitutes an atomic operation in Python, anyway?
>
>


Well, although only one thread can run at a time due to the GIL you
can't accurately predict when the GIL is going to be released and
therefore you don't know when another thread is going to pick up and
start going (GIL is released on every so many byte instructs - correct
me if i'm wrong, certain operations that have a bit to do with IO, and
modules you wrote yourself where you manually release the GIL using
macros provided in the C API). If you have your own data structure that
is shared among threads you can use the threading modules
synchronization constructs to get the job done, using locks, conditions,
and events. Queue.Queue is also a good way to go about communicating
with threads.



> e.g.
>
> class IoThread:
> # ...
>
> # called from the other thread...
> def RequestFile(self, name):
> self.fileQueue.append(name)
>
> # called during the IO thread
> def GetNextFile(self);
> next = self.fileQueue[0]
> self.fileQueue.pop(0)
> return next
>
> ?
> -tom!
>



--

Carl J. Van Arsdall
(E-Mail Removed)
Build and Release
MontaVista Software

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
To be not, or not to be not? Ruby Freak Ruby 2 09-23-2008 08:04 AM
Why not 'foo = not f' instead of 'foo = (not f or 1) and 0'? Kristian Domke Python 11 01-23-2008 07:27 PM
'' is not a valid name. Make sure that it does not include invalid characters or punctuation and that it is not too long. rote ASP .Net 2 01-23-2008 03:07 PM
Cisco 3640 3620 3600 not detecting, not enabling, not working: NM-2FE2W Taki Soho Cisco 0 09-22-2004 07:28 AM
maintaining control with cookies (not strictly an ASP or even server side question. But not not either) Stephanie Stowe ASP General 2 04-07-2004 04:23 PM



Advertisments