Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Executing a python script while it is running

Reply
Thread Tools

Re: Executing a python script while it is running

 
 
Zach Hobesh
Guest
Posts: n/a
 
      06-16-2009
> A lot more information would be useful. *What version of Python, and what
> operating system environment? *Exactly what would you like to happen when
> the batch file is invoked a second time?


I'm running Python 2.6.2 on Windows. I'm passing filenames to the
batch files and I need all filenames to be processed. I can't have
any fails. I'm working on logging any fails I do have so that I can
maybe batch process at the end of the day.

> *2) let them both run as separate processes


This sounds like a good option, but I'm not totally sure on how to go
about this?

> *4) queue something to be processed when the first run finishes


I had the same idea, but I believe it would involve having another
python script run all day long, which wouldn't necessarily be a bad
thing, but I'd like to explore other options as well.

> What provisions does this existing application have for long-running batch
> files? *Seems the synchronization ought to happen there. *Do you have any
> constraints on how long your script might take, worst case? *What if the
> application finishes its tasks at a faster average rate than your script can
> process them?


The batch file is moving large video files. Duration probably ranges
from 10 sec to 45 mins. On average, the application takes longer to
process the files than it does the batch file/python script takes to
copy them, but I'm concerned about the occasional time that the
application finishes a small file right after finishing a large file.

Thanks for your response!

-Zach
 
Reply With Quote
 
 
 
 
Lie Ryan
Guest
Posts: n/a
 
      06-17-2009
Zach Hobesh wrote:
>> A lot more information would be useful. What version of Python, and what
>> operating system environment? Exactly what would you like to happen when
>> the batch file is invoked a second time?

>
> I'm running Python 2.6.2 on Windows. I'm passing filenames to the
> batch files and I need all filenames to be processed. I can't have
> any fails. I'm working on logging any fails I do have so that I can
> maybe batch process at the end of the day.
>
>> 2) let them both run as separate processes

>
> This sounds like a good option, but I'm not totally sure on how to go
> about this?


For that one, you don't really have to do anything special as long as
both program doesn't try to modify the same file at the same time. If
the two program need to modify the same file, you need to arrange some
coordination, the specifics of which highly depends on what you're
trying to do.

>> 4) queue something to be processed when the first run finishes

>
> I had the same idea, but I believe it would involve having another
> python script run all day long, which wouldn't necessarily be a bad
> thing, but I'd like to explore other options as well.


You don't necessarily have to have daemon process, cron/scheduled task
can do it as well.


5) detect for another process; if there is, sleep until the it
terminates, then do the job. If there is a good possibility of 3 or more
process, it might be necessary to use a lock file, if the lock file
exists, sleep, else create a lock file and do the job.

>> What provisions does this existing application have for long-running batch
>> files? Seems the synchronization ought to happen there. Do you have any
>> constraints on how long your script might take, worst case? What if the
>> application finishes its tasks at a faster average rate than your script can
>> process them?

>
> The batch file is moving large video files. Duration probably ranges
> from 10 sec to 45 mins. On average, the application takes longer to
> process the files than it does the batch file/python script takes to
> copy them, but I'm concerned about the occasional time that the
> application finishes a small file right after finishing a large file.
>
> Thanks for your response!
>
> -Zach

 
Reply With Quote
 
 
 
 
Aaron Brady
Guest
Posts: n/a
 
      06-17-2009
On Jun 16, 3:48*pm, Zach Hobesh <(E-Mail Removed)> wrote:
> > A lot more information would be useful. *What version of Python, and what
> > operating system environment? *Exactly what would you like to happen when
> > the batch file is invoked a second time?

>
> I'm running Python 2.6.2 on Windows. *I'm passing filenames to the
> batch files and I need all filenames to be processed. *I can't have
> any fails. *I'm working on logging any fails I do have so that I can
> maybe batch process at the end of the day.
>
> > *2) let them both run as separate processes

>
> This sounds like a good option, but I'm not totally sure on how to go
> about this?
>
> > *4) queue something to be processed when the first run finishes

>
> I had the same idea, but I believe it would involve having another
> python script run all day long, which wouldn't necessarily be a bad
> thing, but I'd like to explore other options as well.


This sort of falls under both categories, 2 & 4, and it will probably
be judged 'poor practice' by history. We're all historians now, I
guess.

Windows has what's called a 'named mutex' for interprocess
synchro'tion. Start your new process, acquire their shared mutex by
name, and block on it. You will have one process for each file, but
only one will run at once.

You won't even need to build a shared library; 'ctypes' will suffice.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Passing arguments to & executing, a python script on a remote machinefrom a python script on local machine ashish Python 4 09-20-2012 07:28 AM
Executing a python script while it is running Zach Hobesh Python 0 06-16-2009 09:36 PM
modify a long-running python script while it is running? Benjamin Rutt Python 2 12-20-2005 01:42 PM
Executing a python script with arguments from a python script olaufr@gmail.com Python 3 12-13-2005 02:49 AM
executing a python script from another python script. sarmin kho Python 1 06-04-2004 01:12 PM



Advertisments