Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > scheduler or infinite loop

Reply
Thread Tools

scheduler or infinite loop

 
 
harryos
Guest
Posts: n/a
 
      09-29-2010
hi
I am trying to write a program to read data from a site url.
The program must read the data from site every 5 minutes.

def get_data_from_site(pageurlstr):
h=urllib.urlopen(pageurlstr)
data=h.read()
process_data(data)

At first, I thought of using the sched module ,but then it doesn't
look right adding so many scheduler.enter() statements.The program is
supposed to execute the above function every
5 minutes until the application is shut down by the user.

I created an infinite loop
while True:
print time.asctime()
get_data_from_site('http://somesite.com/')
time.sleep(300)

Is there a better way to do this?Any suggestions ,pointers most
welcome
thanks
harry
 
Reply With Quote
 
 
 
 
Frank Millman
Guest
Posts: n/a
 
      09-29-2010

harryos wrote:

> hi
> I am trying to write a program to read data from a site url.
> The program must read the data from site every 5 minutes.
>
> def get_data_from_site(pageurlstr):
> h=urllib.urlopen(pageurlstr)
> data=h.read()
> process_data(data)
>
> At first, I thought of using the sched module ,but then it doesn't
> look right adding so many scheduler.enter() statements.The program is
> supposed to execute the above function every
> 5 minutes until the application is shut down by the user.
>
> I created an infinite loop
> while True:
> print time.asctime()
> get_data_from_site('http://somesite.com/')
> time.sleep(300)
>
> Is there a better way to do this?Any suggestions ,pointers most
> welcome
> thanks
> harry


Here is a technique that allows the loop to run in the background, in its
own thread, leaving the main program to do other processing -

import threading

class DataGetter(threading.Thread):

def __init__(self):
threading.Thread.__init__(self)
self.event = threading.Event()

def run(self):
event = self.event # make local
while not event.is_set():
print time.asctime()
# either call the function from here,
# or put the body of the function here
get_data_from_site('http://somesite.com/')
event.wait(300)

def stop(self):
self.event.set()

In the startup of the main program, insert the following -

data_getter = DataGetter()
data_getter.start()

At the end of the program, terminate the loop like this -
data_getter.stop()

HTH

Frank Millman


 
Reply With Quote
 
 
 
 
harryos
Guest
Posts: n/a
 
      09-29-2010
thanks Frank
>
> Here is a technique that allows the loop to run in the background, in its
> own thread, leaving the main program to do other processing -
>
> import threading
>
> class DataGetter(threading.Thread):
>

 
Reply With Quote
 
Frank Millman
Guest
Posts: n/a
 
      09-30-2010
harryos wrote
>>
>> Here is a technique that allows the loop to run in the background, in its
>> own thread, leaving the main program to do other processing -
>>
>> import threading
>>
>> class DataGetter(threading.Thread):
>>


> thanks Frank


A pleasure.

I left out a line that will usually be desirable.

At the end of the program, you should have -

data_getter.stop()
+ data_getter.join()

This forces the main program to wait until the thread has terminated before
continuing.

Frank


 
Reply With Quote
 
John Nagle
Guest
Posts: n/a
 
      09-30-2010
On 9/29/2010 4:59 AM, harryos wrote:
> hi
> I am trying to write a program to read data from a site url.
> The program must read the data from site every 5 minutes.
>
> def get_data_from_site(pageurlstr):
> h=urllib.urlopen(pageurlstr)
> data=h.read()
> process_data(data)


A key point here is that you're not handling network
failures. The first time you have a brief network
outage, your program will fail.

Consider something like this:



def get_data_from_site(pageurlstr):
try :
h=urllib.urlopen(pageurlstr)
data=h.read()
except EnvironmentError as message :
print("Error reading %s from network at %s: %s" %
(pageurlstr, time.asctime(), str(message))
return(False)
process_data(data)
return(True)


lastpoll = 0 # time of last successful read
POLLINTERVAL = 300.0
RETRYINTERVAL = 30.0
while True:
status = get_data_from_site('http://somesite.com/')
if not status : # if fail
time.sleep(RETRYINTERVAL) # try again soon
print("Retrying...")
continue
now = time.time() # success
# Wait for the next poll period. Compensate for how
# long the read took.
waittime = max(POLLINTERVAL - (now - lastpoll), 1.0)
lastpoll = now
time.sleep(waittime)





 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Triple nested loop python (While loop insde of for loop inside ofwhile loop) Isaac Won Python 9 03-04-2013 10:08 AM
Dotnet scheduler like the Windows "Schedued Tasks" scheduler? Codemonkey ASP .Net 8 06-10-2011 10:53 AM
infinite loop unexpectly dies Vedran Vukotic Perl 0 03-02-2006 07:43 AM
loop thru a STL list causes an infinite loop Allerdyce.John@gmail.com C++ 5 01-31-2006 03:21 PM
Infinite loop when using Server.Transfer Alexander Bosch ASP .Net 11 11-10-2004 08:53 AM



Advertisments