Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > cgi concurrency approaches?

Reply
Thread Tools

cgi concurrency approaches?

 
 
Paul Rubin
Guest
Posts: n/a
 
      01-23-2004
I'm wondering if folks here have favorite lightweight ways of dealing
with concurrency in cgi's. Take a simple case:

You want to write a cgi that implements a simple counter. The first
time it's called, it prints "1". The next time, "2", etc. The naive
implementation looks something like:

fd = open("counter.num", "rw")
n = int(fd.read())
fd.seek(0, 0)
fd.write("%d"% n+1)
print "Content-type: text/html\n\n"
print n

but of course there's the obvious race condition if two people hit the
cgi at the same time.

Fancier solutions include running an transactional database in another
process and connecting to it, setting up a daemon that remembers the
counter value in memory and serializes access through a socket that
the cgi opens, using a file-system specific hack like linking to
counter file to lock it, having a timeout/retry if the counter is
locked, with a possible hangup if a lock file gets left around by
accident, etc. Each is a big pain in the neck.

Anyone have some simpler approaches?
 
Reply With Quote
 
 
 
 
Josiah Carlson
Guest
Posts: n/a
 
      01-23-2004
Paul Rubin wrote:

> I'm wondering if folks here have favorite lightweight ways of dealing
> with concurrency in cgi's. Take a simple case:
>
> You want to write a cgi that implements a simple counter. The first
> time it's called, it prints "1". The next time, "2", etc. The naive
> implementation looks something like:
>
> fd = open("counter.num", "rw")
> n = int(fd.read())
> fd.seek(0, 0)
> fd.write("%d"% n+1)
> print "Content-type: text/html\n\n"
> print n
>
> but of course there's the obvious race condition if two people hit the
> cgi at the same time.
>
> Fancier solutions include running an transactional database in another
> process and connecting to it, setting up a daemon that remembers the
> counter value in memory and serializes access through a socket that
> the cgi opens, using a file-system specific hack like linking to
> counter file to lock it, having a timeout/retry if the counter is
> locked, with a possible hangup if a lock file gets left around by
> accident, etc. Each is a big pain in the neck.
>
> Anyone have some simpler approaches?


I'd just run Xitami (www.xitami.org) and connect a LRWP (Long-Running
Web Process). I'm not familliar with Apache, so I don't know if
mod_python is equivalent.

- Josiah





 
Reply With Quote
 
 
 
 
Rene Pijlman
Guest
Posts: n/a
 
      01-23-2004
Paul Rubin <http://(E-Mail Removed)>:
>I'm wondering if folks here have favorite lightweight ways of dealing
>with concurrency in cgi's. Take a simple case:
>
>You want to write a cgi that implements a simple counter. The first
>time it's called, it prints "1". The next time, "2", etc.

[...]
>Anyone have some simpler approaches?


Use mod_python and keep the counter in a Python variable (if the value
needs not be persistent).
http://www.modpython.org/

--
René Pijlman
 
Reply With Quote
 
Matt Goodall
Guest
Posts: n/a
 
      01-23-2004
On Fri, 2004-01-23 at 10:04, Rene Pijlman wrote:
> Paul Rubin <http://(E-Mail Removed)>:
> >I'm wondering if folks here have favorite lightweight ways of dealing
> >with concurrency in cgi's. Take a simple case:
> >
> >You want to write a cgi that implements a simple counter. The first
> >time it's called, it prints "1". The next time, "2", etc.

> [...]
> >Anyone have some simpler approaches?

>
> Use mod_python and keep the counter in a Python variable (if the value
> needs not be persistent).
> http://www.modpython.org/


That will only work if you use Apache 2 in threaded mode. Apache 1 and
Apache 2 in forking mode (multiple processes) will have a copy of the
counter object per-process.

Cheers, Matt

--
Matt Goodall, Pollenation Internet Ltd
w: http://www.pollenation.net
e: http://www.velocityreviews.com/forums/(E-Mail Removed)

Any views expressed are my own and do not necessarily reflect the
views of my employer.


 
Reply With Quote
 
Rene Pijlman
Guest
Posts: n/a
 
      01-23-2004
Matt Goodall:
>Rene Pijlman:
>> Use mod_python and keep the counter in a Python variable

>
>That will only work if you use Apache 2 in threaded mode.


Of course, good point.

--
René Pijlman
 
Reply With Quote
 
Alan Kennedy
Guest
Posts: n/a
 
      01-23-2004
[replying to someone else's reply because I missed the original]

[Paul Rubin]
> I'm wondering if folks here have favorite lightweight ways of dealing
> with concurrency in cgi's,

[snip]
> but of course there's the obvious race
> condition if two people hit the
> cgi at the same time.
>
> Fancier solutions include running an transactional database in another
> process and connecting to it, setting up a daemon that remembers the
> counter value in memory and serializes access through a socket that
> the cgi opens, using a file-system specific hack like linking to
> counter file to lock it, having a timeout/retry if the counter is
> locked, with a possible hangup if a lock file gets left around by
> accident, etc. Each is a big pain in the neck.
> Anyone have some simpler approaches?


I don't know about a platform independent solution, but if you were
willing to limit yourself to certain varieties of Unix, e.g. Linux, it
would probably be relatively easy to implement a persistent counter
using something like System V message queues, or shared memory +
semaphores. This obviously will work only on systems which support
System V IPC.

Here is a brief page about System V IPC

http://www.science.unitn.it/~fiorell...lk/node56.html

And here's a module, which I've never used, which purports to provide
a python interface to System V IPC facilities, if available.

http://www.theory.org/~mac4/software/

Given the above facilities, there are two ways I would approach the
problem of a persistent counter.

1. Use a dedicated Queue, and hold the counter value inside a single
message which "lives" on that queue. Those incrementing the counter
read the single message from the queue, increment it and put it back
on the queue. Any processes awaiting access to the "counter message"
would then do a blocking read on the queue. Since all puts and gets of
messages are atomic, you are guaranteed only atomic updates to your
counter. However, you could lock your system up if one of your
accessing CGI processes did not put the counter back again! You can
use timeouts as well, if my memory serves (it's been a long time since
I used System V IPC). You can also get fancy, with priorities, etc.

2. Store the value in pre-created shared memory partition, protected
by a semaphore. Since there are shared memory python modules for
several platforms, you might have a better chance with this approach
on non-unix platforms.

The Python Object Sharing (POSH) module might provide wrappers to some
useful functionality, although the module itself might be a little
heavyweight for providing a simple persistent counter.

http://poshmodule.sourceforge.net/posh/html/posh.html

HTH,

--
alan kennedy
------------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/contact/alan
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
(CGI-Target)Could not connect to CGI-Proxy John Smith Java 0 05-15-2006 09:21 PM
Python-cgi or Perl-cgi script doubt praba kar Python 1 07-30-2005 08:25 AM
Python CGI - Accepting Input, Invoking Another Process, Ending CGI LarsenMTL Python 4 11-04-2004 05:59 PM
Calling cgi from cgi thru 'system' function. Different behaviour on browser v/s cmd line Shailan Perl 2 12-15-2003 04:26 PM
Re: CGI Perl "use CGI" statement fail Jürgen Exner Perl 0 07-31-2003 02:00 PM



Advertisments