Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: FW: [Tutor] Multi-Threading and KeyboardInterrupt

Reply
Thread Tools

Re: FW: [Tutor] Multi-Threading and KeyboardInterrupt

 
 
Dennis Lee Bieber
Guest
Posts: n/a
 
      06-12-2009
On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
<(E-Mail Removed)> declaimed the following in
gmane.comp.python.general:

> I sent this to the Tutor mailing list and did not receive a response.
> Perhaps one of you might be able to offer some sagely wisdom or pointed
> remarks?
>
> Please reply off-list and thanks in advance. Code examples are below in
> plain text.
>

Sorry -- you post to a public forum, expect to get the response on a
public forum...

> > My program runs interactively by allowing the user to directly
> > interact with the python prompt. This program has a runAll() method
> > that runs a series of subprocesses with a cap on how many instances
> > are running at a time. My intent is to allow the user to use Ctrl-C to
> > break these subprocesses. Note that while not reflected in the demo


Are they subprocesses or threads? Your sample code seems to be using
threads.

When using threads, there is no assurance that any thread other than
the main program will receive a keyboard interrupt. If true
subprocesses, they may see a keyboard interrupt IF stdin is connected to
that subprocess AND the subprocess is the current active process when
the keyboard interrupt was detected.

> def runAll():
> workers = [ Thread(target = runSingle, args = [i])
> for i in xrange(MAX_SUBPROCS + 1) ]
> try:
> for w in workers:
> w.start()
> except KeyboardInterrupt:
> ## I want this to be shown on a KeyboardInterrupt
> print '********* stopped midway ********'


You are unlikely to see that... After you start the defined worker
/threads/ (which doesn't take very long -- all threads will be started,
but some may immediately block on the semaphore) this block will exit
and you will be at...

> for w in workers:
> w.join()
>

... a .join() call, which is the most likely position at which the
keyboard interrupt will be processed, killing the main program thread
and probably generating some errors as dangling active threads are
forceably killed.
--
Wulfraed Dennis Lee Bieber KD6MOG
http://www.velocityreviews.com/forums/(E-Mail Removed) (E-Mail Removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (E-Mail Removed))
HTTP://www.bestiaria.com/

 
Reply With Quote
 
 
 
 
Graham Dumpleton
Guest
Posts: n/a
 
      06-12-2009
On Jun 12, 3:35*pm, Dennis Lee Bieber <(E-Mail Removed)> wrote:
> On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
> <(E-Mail Removed)> declaimed the following in
> gmane.comp.python.general:
>
> > I sent this to the Tutor mailing list and did not receive a response.
> > Perhaps one of you might be able to offer some sagely wisdom or pointed
> > remarks?

>
> > Please reply off-list and thanks in advance. Code examples are below in
> > plain text.

>
> * * * * Sorry -- you post to a public forum, expect to get the response on a
> public forum...


Bbit off topic, but if you are a proponent of public forums, how come
your post on Google Groups shows:

"""Note: The author of this message requested that it not be archived.
This message will be removed from Groups in 6 days (Jun 19, 3:35
pm)."""

I always find it a tad annoying that people have this, as you loose
posts from the record of a discussion and if that post carries useful
information, it is lost.

Am also curious as to what mechanism for posting you use that allows
you to set an expiration time on the post anyway. Are you using old
usenet news reader or something?

Graham
 
Reply With Quote
 
 
 
 
Dennis Lee Bieber
Guest
Posts: n/a
 
      06-13-2009
On Thu, 11 Jun 2009 23:29:10 -0700 (PDT), Graham Dumpleton
<(E-Mail Removed)> declaimed the following in
gmane.comp.python.general:

> On Jun 12, 3:35*pm, Dennis Lee Bieber <(E-Mail Removed)> wrote:
> > On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
> > <(E-Mail Removed)> declaimed the following in
> > gmane.comp.python.general:
> >
> > > I sent this to the Tutor mailing list and did not receive a response.
> > > Perhaps one of you might be able to offer some sagely wisdom or pointed
> > > remarks?

> >
> > > Please reply off-list and thanks in advance. Code examples are below in
> > > plain text.

> >
> > * * * * Sorry -- you post to a public forum, expect to get the response on a
> > public forum...

>
> Bbit off topic, but if you are a proponent of public forums, how come
> your post on Google Groups shows:
>
> """Note: The author of this message requested that it not be archived.
> This message will be removed from Groups in 6 days (Jun 19, 3:35
> pm)."""
>
> I always find it a tad annoying that people have this, as you loose
> posts from the record of a discussion and if that post carries useful
> information, it is lost.
>
> Am also curious as to what mechanism for posting you use that allows
> you to set an expiration time on the post anyway. Are you using old
> usenet news reader or something?
>


While I've not upgraded it recently, I wouldn't call it "old"...

And yes, what else would I use to access three different NNTP
servers -- downloading messages in the background... Posting all replies
in a batch prior to my shutting down for the night...

A web browser where I have to give it all my attention for each
message? Peruse all the unfiltered junk that shows up? And have to guess
which topics have new messages? With practically no editing control, and
no capability to hold a draft reply for later...

As for the use of X-Noarchive headers... Back before Google, most
NNTP servers had automatic expiration of messages (on my ISP at the
time, the binary groups expired after 24 hours!, most messages expired
after one or two weeks). I never used X-Noarchive -- in truth, I didn't
even know about it.

THEN... Google started doing Google Groups with no automatic
expiration. That is when the subject of X-Noarchive headers came up in
some of the groups I'd been following. I and a fair number of others
immediately took advantage of it.
--
Wulfraed Dennis Lee Bieber KD6MOG
(E-Mail Removed) (E-Mail Removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (E-Mail Removed))
HTTP://www.bestiaria.com/

 
Reply With Quote
 
Mike Kazantsev
Guest
Posts: n/a
 
      06-13-2009
On Thu, 11 Jun 2009 22:35:15 -0700
Dennis Lee Bieber <(E-Mail Removed)> wrote:

> On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
> <(E-Mail Removed)> declaimed the following in
> gmane.comp.python.general:
>
> > I sent this to the Tutor mailing list and did not receive a response.
> > Perhaps one of you might be able to offer some sagely wisdom or pointed
> > remarks?
> >
> > Please reply off-list and thanks in advance. Code examples are below in
> > plain text.
> >

> Sorry -- you post to a public forum, expect to get the response on a
> public forum...
>
> > > My program runs interactively by allowing the user to directly
> > > interact with the python prompt. This program has a runAll() method
> > > that runs a series of subprocesses with a cap on how many instances
> > > are running at a time. My intent is to allow the user to use Ctrl-C to
> > > break these subprocesses. Note that while not reflected in the demo

>
> Are they subprocesses or threads? Your sample code seems to be using
> threads.
>
> When using threads, there is no assurance that any thread other than
> the main program will receive a keyboard interrupt.


In fact, no thread other than the main will get interrupt.


> > def runAll():
> > workers = [ Thread(target = runSingle, args = [i])
> > for i in xrange(MAX_SUBPROCS + 1) ]
> > try:
> > for w in workers:
> > w.start()
> > except KeyboardInterrupt:
> > ## I want this to be shown on a KeyboardInterrupt
> > print '********* stopped midway ********'

>
> You are unlikely to see that... After you start the defined worker
> /threads/ (which doesn't take very long -- all threads will be started,
> but some may immediately block on the semaphore) this block will exit
> and you will be at...
>
> > for w in workers:
> > w.join()

>
> ... a .join() call, which is the most likely position at which the
> keyboard interrupt will be processed, killing the main program thread
> and probably generating some errors as dangling active threads are
> forceably killed.


There was quite interesting explaination of what happens when you send
^C with threads, posted on concurrency-sig list recently:

http://blip.tv/file/2232410
http://www.dabeaz.com/python/GIL.pdf

Can be quite shocking, but my experience w/ threads only confirms that.


--
Mike Kazantsev // fraggod.net

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkozOF0ACgkQASbOZpzyXnGzZQCfftmsS/fraJ/XYe9r915gmU4I
negAmwdiSIH6gtYfbvZ/R5DDxuUF5O6d
=oNGI
-----END PGP SIGNATURE-----

 
Reply With Quote
 
koranthala
Guest
Posts: n/a
 
      06-13-2009
On Jun 13, 10:25*am, Mike Kazantsev <(E-Mail Removed)> wrote:
> On Thu, 11 Jun 2009 22:35:15 -0700
> Dennis Lee Bieber <(E-Mail Removed)> wrote:
>
>
>
> > On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
> > <(E-Mail Removed)> declaimed the following in
> > gmane.comp.python.general:

>
> > > I sent this to the Tutor mailing list and did not receive a response.
> > > Perhaps one of you might be able to offer some sagely wisdom or pointed
> > > remarks?

>
> > > Please reply off-list and thanks in advance. Code examples are below in
> > > plain text.

>
> > * *Sorry -- you post to a public forum, expect to get the response on a
> > public forum...

>
> > > > My program runs interactively by allowing the user to directly
> > > > interact with the python prompt. This program has a runAll() method
> > > > that runs a series of subprocesses with a cap on how many instances
> > > > are running at a time. My intent is to allow the user to use Ctrl-C to
> > > > break these subprocesses. Note that while not reflected in the demo

>
> > * *Are they subprocesses or threads? Your sample code seems to be using
> > threads.

>
> > * *When using threads, there is no assurance that any thread other than
> > the main program will receive a keyboard interrupt.

>
> In fact, no thread other than the main will get interrupt.
>
>
>
> > > def runAll():
> > > * * workers = [ Thread(target = runSingle, args = [i])
> > > * * * * * * for i in xrange(MAX_SUBPROCS + 1) ]
> > > * * try:
> > > * * * * for w in workers:
> > > * * * * * * w.start()
> > > * * except KeyboardInterrupt:
> > > * * * * ## I want this to be shown on a KeyboardInterrupt
> > > * * * * print '********* stopped midway ********'

>
> > * *You are unlikely to see that... After you start the defined worker
> > /threads/ (which doesn't take very long -- all threads will be started,
> > but some may immediately block on the semaphore) this block will exit
> > and you will be at...

>
> > > * * for w in workers:
> > > * * * * w.join()

>
> > * *... a .join() call, which is the most likely position at which the
> > keyboard interrupt will be processed, killing the main program thread
> > and probably generating some errors as dangling active threads are
> > forceably killed.

>
> There was quite interesting explaination of what happens when you send
> ^C with threads, posted on concurrency-sig list recently:
>
> *http://blip.tv/file/2232410
> *http://www.dabeaz.com/python/GIL.pdf
>
> Can be quite shocking, but my experience w/ threads only confirms that.
>
> --
> Mike Kazantsev // fraggod.net
>
> *signature.asc
> < 1KViewDownload


Thank you very much for the link Mike.
Are there other videos/audio like this? I am learning more from these
videos than by experience alone.
I did find one - http://www.awaretek.com/python/ - are there other
links?
 
Reply With Quote
 
Mike Kazantsev
Guest
Posts: n/a
 
      06-13-2009
On Sat, 13 Jun 2009 04:42:16 -0700 (PDT)
koranthala <(E-Mail Removed)> wrote:

> Are there other videos/audio like this? I am learning more from these
> videos than by experience alone.


Indeed, it is a very interesting presentation, but I'm afraid I've
stumbled upon it just as you did, but on concurrency-sig mailing list.

It's a relatively new list (now hosted on mail.python.org), not
specifically dedicated to podcasts or, for that matter, any
implementation details. I haven't seen any other material like this
there.

> I did find one - http://www.awaretek.com/python/ - are there other
> links?


Thanks for sharing this link, although I prefer such information in
written form - it's easier/faster to work with and much more accessible.

--
Mike Kazantsev // fraggod.net

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkoznS4ACgkQASbOZpzyXnE/7wCgrRLpSZy7tASinsNabZMyPzdt
pQQAoKMeS7I9dmbvYAqg9JOwHjjSgxmY
=d89p
-----END PGP SIGNATURE-----

 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      06-14-2009
In article <(E-Mail Removed)>,
Dennis Lee Bieber <(E-Mail Removed)> wrote:
>
> As for the use of X-Noarchive headers... Back before Google, most
>NNTP servers had automatic expiration of messages (on my ISP at the
>time, the binary groups expired after 24 hours!, most messages expired
>after one or two weeks). I never used X-Noarchive -- in truth, I didn't
>even know about it.
>
> THEN... Google started doing Google Groups with no automatic
>expiration. That is when the subject of X-Noarchive headers came up in
>some of the groups I'd been following. I and a fair number of others
>immediately took advantage of it.


Actually, it's not really fair to blame Google for this, they were only
following the design of DejaNews, which they purchased in 2001. Google
did create some additional value by finding old Usenet archives that
predated DejaNews. (Google Groups, which I normally refer to as Gooja in
deference to the history, debuted the archive on Dec 10, 2001.)
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
 
Reply With Quote
 
Dennis Lee Bieber
Guest
Posts: n/a
 
      06-14-2009
On 13 Jun 2009 21:22:58 -0700, (E-Mail Removed) (Aahz) declaimed the
following in gmane.comp.python.general:

> Actually, it's not really fair to blame Google for this, they were only
> following the design of DejaNews, which they purchased in 2001. Google
> did create some additional value by finding old Usenet archives that
> predated DejaNews. (Google Groups, which I normally refer to as Gooja in
> deference to the history, debuted the archive on Dec 10, 2001.)


Mea culpa -- I had a feeling there was something just before
Google... But they sort of made it really visible <G>

Either way -- it was still a change from "expiration at some
date"... Though since (Netcom/Mindspring)Earthlink seems to have
subcontracted NNTP service to Giganews (or some such) it wouldn't
surprise me to learn that service also keeps a mammoth archive...
--
Wulfraed Dennis Lee Bieber KD6MOG
(E-Mail Removed) (E-Mail Removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (E-Mail Removed))
HTTP://www.bestiaria.com/

 
Reply With Quote
 
David Bolen
Guest
Posts: n/a
 
      06-14-2009
Dennis Lee Bieber <(E-Mail Removed)> writes:

> Either way -- it was still a change from "expiration at some
> date"... Though since (Netcom/Mindspring)Earthlink seems to have
> subcontracted NNTP service to Giganews (or some such) it wouldn't
> surprise me to learn that service also keeps a mammoth archive...


I'm not sure it's really a change, or if it is, it certainly isn't a
change from how things were originally. "Expiration at some date" was
never any sort of global policy for Usenet - just an aspect of a
individual news server. Some servers held messages for long periods,
particularly for the big seven groups - it's true that alt.* and in
particular the binaries, might expire quickly. I know I certainly ran
some servers that didn't bother expiring - or had expiration times in
years - of the big seven. My experience post-dates the great
renaming, so I can't speak to before that, but don't think behavior
was very different.

Individual messages could include an Expires: header if they wished,
but even that was just a suggestion. Any actual expiration was due to
local configuration on each news server, which while it could take
Expires: headers into account, was just as often driven by local
storage availability or the whims of the local news admin

I think Deja News was providing web access to their archive from the
mid-90s on (so quite a while before Google even existed) so certainly
by that point everyone had access to a rather complete archive even if
messages had expired on their local server. I think Deja was also the
first to introduce X-No-Archive. But other archives certainly existed
pre-Deja, which I'm sure is, in large part, how Google was able to
locate and incorporate the older messages into their system after
their acquisition of the Deja archive.

-- David
 
Reply With Quote
 
David Bolen
Guest
Posts: n/a
 
      06-14-2009
Ben Finney <(E-Mail Removed)> writes:

> David Bolen <(E-Mail Removed)> writes:
>
>> Individual messages could include an Expires: header if they wished,

>
> Since we're already well off-topic: NNTP, HTTP, and email, and probably
> other protocols as well, all deal with messages. They are all consistent
> in defining a message [0] as having *exactly one* header.


Heh, I'm not sure it's quite as consistent as you may think,
particularly with older RFCs, which are relevant in this discussion
since we're talking about historical artifacts.

For example, while more recent mail RFCs like 2822 may specifically
talk about header fields as the "header" (singular) of the message,
the older RFC 822 instead refers to a "headers" (plural) section.

> Every time you call a field from the header “a header”, or refer to
> the plural “headers of a message”, the IETF kills a kitten. You
> don't want to hurt a kitten, do you?


Heaven forbid - though I'd think I could hold my own with the IETF.
My reference to "header" was in lieu of "header line", something that
the Usenet RFCs (1036, and the older 850) do extensively themselves.

But I'll be more careful in the future - need to ensure kitten safety!

-- David
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: exception KeyboardInterrupt and os.system command darren kirby Python 1 11-29-2005 08:26 AM
Re: exception KeyboardInterrupt and os.system command jepler@unpythonic.net Python 3 11-28-2005 06:21 PM
exception KeyboardInterrupt and os.system command darren kirby Python 1 11-27-2005 11:18 PM
Socket object and KeyboardInterrupt exception PantherSE Python 0 05-16-2005 09:10 PM
KeyboardInterrupt and threading Ivan Nestlerode Python 8 01-08-2004 11:12 AM



Advertisments