Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Question about long-running web scripts

Reply
Thread Tools

Question about long-running web scripts

 
 
Gilles
Guest
Posts: n/a
 
      10-25-2012
Hello

I'd like to check something about running Python web applications.

Generally speaking, the reason scripts run faster when called through
FastCGI or the mod_* modules, is because the interpreter is already up
and running.
But when running PHP scripts, this does nothing about fetching the
file from disk, recompiling, rerunning it, and usually reconnecting to
the database.

OTOH, Python web scripts can be written as long-running scripts: In
this case, what is the added-value of using FastCGI? Why can't the web
server simply call the Python script directly, just like CGI?

Thank you.
 
Reply With Quote
 
 
 
 
Tim Golden
Guest
Posts: n/a
 
      10-25-2012
On 25/10/2012 12:45, Gilles wrote:
> I'd like to check something about running Python web applications.
>
> Generally speaking, the reason scripts run faster when called
> through FastCGI or the mod_* modules, is because the interpreter is
> already up and running. But when running PHP scripts, this does
> nothing about fetching the file from disk, recompiling, rerunning it,
> and usually reconnecting to the database.
>
> OTOH, Python web scripts can be written as long-running scripts: In
> this case, what is the added-value of using FastCGI? Why can't the
> web server simply call the Python script directly, just like CGI?


(Your question is a little confused at the end. I'm choosing to
understand: why can't we just run Python one-shot, like CGI? The likely
alternative meaning is: why can't the incoming request be routed to an
already-running Python program -- which is not, of course, what CGI
generally does. Hence my confusion).

The answer is: it can. CGI is a protocol rather than anything else. In
front of a CGI exchange is the browser (or some other web client).
Behind it is some program which is capable of producing a valid HTTP
response, including a Python program.

It's perfectly possible to run a usable website against Python running
one-shot. You won't get terrific performance out of it, but for a
website which doesn't expect humungous amounts of traffic, it'll work fine.

The amount of time it takes a half-decent, even shared, server to start
up a Python process, connect to a database, pull stuff together, and
send a response will likely not impact on an average user's experience.
As long as too many of them don't try to do that at the same time.
Exactly where the line is drawn will depend on your particular hosting
solution, your assumed traffic, and your users' expectations as to
responsiveness.

TJG
 
Reply With Quote
 
 
 
 
Gilles
Guest
Posts: n/a
 
      10-25-2012
On Thu, 25 Oct 2012 13:03:14 +0100, Tim Golden <(E-Mail Removed)>
wrote:
>(Your question is a little confused at the end. I'm choosing to
>understand: why can't we just run Python one-shot, like CGI? The likely
>alternative meaning is: why can't the incoming request be routed to an
>already-running Python program -- which is not, of course, what CGI
>generally does. Hence my confusion).


Yes indeed. Sorry about the confusion.

But actually, I didn't mean one-shot scripts, where the Python
interpreter + script must be loaded each time, but rather: If I leave
a Python running in an endless loop, why not just use either CGI or
some other basic way to call the script instead of FastCGI?

In the case of PHP, FastCGI makes sense, but I don't see the benefit
for long-running Python scripts.
 
Reply With Quote
 
David Hutto
Guest
Posts: n/a
 
      10-25-2012
On Thu, Oct 25, 2012 at 8:03 AM, Tim Golden <(E-Mail Removed)> wrote:
> On 25/10/2012 12:45, Gilles wrote:
>> I'd like to check something about running Python web applications.
>>
>> Generally speaking, the reason scripts run faster when called
>> through FastCGI or the mod_* modules, is because the interpreter is
>> already up and running. But when running PHP scripts, this does
>> nothing about fetching the file from disk, recompiling, rerunning it,
>> and usually reconnecting to the database.
>>


I'd say that is the same as py, unless it's a cron job to limit script
iterations

>> OTOH, Python web scripts can be written as long-running scripts: In
>> this case, what is the added-value of using FastCGI? Why can't the
>> web server simply call the Python script directly, just like CGI?


The server should call a the script, or script.sleep()

There are also server options to setup when a script is run, other
than a cron jo for php.

>
> (Your question is a little confused at the end. I'm choosing to
> understand: why can't we just run Python one-shot, like CGI? The likely
> alternative meaning is: why can't the incoming request be routed to an
> already-running Python program -- which is not, of course, what CGI
> generally does. Hence my confusion).
>
> The answer is: it can. CGI is a protocol rather than anything else. In
> front of a CGI exchange is the browser (or some other web client).
> Behind it is some program which is capable of producing a valid HTTP
> response, including a Python program.
>
> It's perfectly possible to run a usable website against Python running
> one-shot. You won't get terrific performance out of it, but for a
> website which doesn't expect humungous amounts of traffic, it'll work fine.
>
> The amount of time it takes a half-decent, even shared, server to start
> up a Python process, connect to a database, pull stuff together, and
> send a response will likely not impact on an average user's experience.
> As long as too many of them don't try to do that at the same time.
> Exactly where the line is drawn will depend on your particular hosting
> solution, your assumed traffic, and your users' expectations as to
> responsiveness.
>
> TJG
> --
> http://mail.python.org/mailman/listinfo/python-list




--
Best Regards,
David Hutto
CEO: http://www.hitwebdevelopment.com
 
Reply With Quote
 
Tim Golden
Guest
Posts: n/a
 
      10-25-2012
On 25/10/2012 13:40, Gilles wrote:
> On Thu, 25 Oct 2012 13:03:14 +0100, Tim Golden <(E-Mail Removed)>
> wrote:
>> (Your question is a little confused at the end. I'm choosing to
>> understand: why can't we just run Python one-shot, like CGI? The likely
>> alternative meaning is: why can't the incoming request be routed to an
>> already-running Python program -- which is not, of course, what CGI
>> generally does. Hence my confusion).

>
> Yes indeed. Sorry about the confusion.
>
> But actually, I didn't mean one-shot scripts, where the Python
> interpreter + script must be loaded each time, but rather: If I leave
> a Python running in an endless loop, why not just use either CGI or
> some other basic way to call the script instead of FastCGI?


In essence, you're describing FastCGI. A Python program (or, indeed, any
program) which uses FastCGI runs continuously and waits for the incoming
request on a TCP socket (instead of as a sys.stdin stream + env vars
immediately after process startup).

The key description is here:

http://www.fastcgi.com/drupal/node/6?q=node/15

(The sections have no anchors; you're looking for the section titled "2.
FastCGI Interface")

TJG
 
Reply With Quote
 
Gilles
Guest
Posts: n/a
 
      10-26-2012
On Thu, 25 Oct 2012 08:53:11 -0400, David Hutto
<(E-Mail Removed)> wrote:
>>> OTOH, Python web scripts can be written as long-running scripts: In
>>> this case, what is the added-value of using FastCGI? Why can't the
>>> web server simply call the Python script directly, just like CGI?

>
>The server should call a the script, or script.sleep()
>
>There are also server options to setup when a script is run, other
>than a cron jo for php.


Thanks. Does it mean that Python scripts that rely on either fcgi.py
or WSGI really have some endless loop somewhere and will keep running
once they're launched by FastCGI?
 
Reply With Quote
 
Gilles
Guest
Posts: n/a
 
      10-26-2012
On Thu, 25 Oct 2012 14:24:16 +0100, Tim Golden <(E-Mail Removed)>
wrote:
>> But actually, I didn't mean one-shot scripts, where the Python
>> interpreter + script must be loaded each time, but rather: If I leave
>> a Python running in an endless loop, why not just use either CGI or
>> some other basic way to call the script instead of FastCGI?

>
>In essence, you're describing FastCGI. A Python program (or, indeed, any
>program) which uses FastCGI runs continuously and waits for the incoming
>request on a TCP socket (instead of as a sys.stdin stream + env vars
>immediately after process startup).


Thanks for the clarification.

Since, unlike PHP, the Python interpreter is not available in a
FastCGI-capable version, this explains why the www server must be told
which specific Python script to run through FastCGI.

The reason I ask for all this, is that I want to understand how things
work under the hood before relying on a Python framework to handle the
nitty-gritty.
 
Reply With Quote
 
Tim Golden
Guest
Posts: n/a
 
      10-26-2012
On 26/10/2012 10:58, Gilles wrote:
> On Thu, 25 Oct 2012 14:24:16 +0100, Tim Golden <(E-Mail Removed)>
> wrote:
>>> But actually, I didn't mean one-shot scripts, where the Python
>>> interpreter + script must be loaded each time, but rather: If I leave
>>> a Python running in an endless loop, why not just use either CGI or
>>> some other basic way to call the script instead of FastCGI?

>>
>> In essence, you're describing FastCGI. A Python program (or, indeed, any
>> program) which uses FastCGI runs continuously and waits for the incoming
>> request on a TCP socket (instead of as a sys.stdin stream + env vars
>> immediately after process startup).

>
> Thanks for the clarification.
>
> Since, unlike PHP, the Python interpreter is not available in a
> FastCGI-capable version, this explains why the www server must be told
> which specific Python script to run through FastCGI.


I think that this is the distinction you're making:

PHP: mod_php (fastcgi mode) runs myscript.php

Python: <some python fcgi frontend>.py runs myscript.py

which is, essentially, true, not least because PHP and web apps are
pretty much synonymous in many people's minds. Both ways: the only thing
PHP does is web; the simplest route to a web app is PHP.

Certainly there are Python equivalents (mod_python, mod_wsgi, etc.)
which can run in effectively the same way as mod_php, and they could be
configured to run an fcgi frontend script, I presume. There's always a
certain confusion here because you can often one mechanisms (say,
mod_wsgi) to act as another (say legacy one-shot CGI) and because some
things are both mechanism and protocol and it's not always easy to tease
the two apart.


>
> The reason I ask for all this, is that I want to understand how things
> work under the hood before relying on a Python framework to handle the
> nitty-gritty.


Good scheme.

TJG
 
Reply With Quote
 
Gilles
Guest
Posts: n/a
 
      10-26-2012
On Fri, 26 Oct 2012 12:00:17 +0100, Tim Golden <(E-Mail Removed)>
wrote:
>Certainly there are Python equivalents (mod_python, mod_wsgi, etc.)
>which can run in effectively the same way as mod_php, and they could be
>configured to run an fcgi frontend script, I presume. There's always a
>certain confusion here because you can often one mechanisms (say,
>mod_wsgi) to act as another (say legacy one-shot CGI) and because some
>things are both mechanism and protocol and it's not always easy to tease
>the two apart.


Thanks again.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Python configuration question when python scripts are executed usingAppweb as web server. IronyOfLife Python 7 08-11-2009 02:15 AM
Using Python Scripts with IIS - ASP or Python-based CGI scripts withIIS - which makes more sense? davidj411 Python 0 06-27-2008 04:38 PM
What is required for perl scripts to run correct when launched from rc scripts on HPUX 11? deanjones7@gmail.com Perl Misc 13 09-10-2007 11:58 AM
Stupid question: Making scripts python-scripts Jan Danielsson Python 8 07-22-2005 12:20 AM
Re: Stupid question: Making scripts python-scripts Jp Calderone Python 0 07-21-2005 02:38 PM



Advertisments