Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > How to send a var to stdin of an external software

Reply
Thread Tools

How to send a var to stdin of an external software

 
 
Benjamin Watine
Guest
Posts: n/a
 
      03-17-2008
http://www.velocityreviews.com/forums/(E-Mail Removed) a écrit :
> I wrote:
>> And here's a thread example, based on Benjamin's code:

> [...]
>
> Doh! Race condition. Make that:
>
> import subprocess
> import thread
> import Queue
>
> def readtoq(pipe, q):
> q.put(pipe.read())
>
> cat = subprocess.Popen('cat', shell=True, stdin=subprocess.PIPE,
> stdout=subprocess.PIPE)
>
> myVar = str(range(1000000)) # arbitrary test data.
>
> q = Queue.Queue()
> thread.start_new_thread(readtoq, (cat.stdout, q))
> cat.stdin.write(myVar)
> cat.stdin.close()
> cat.wait()
> myNewVar = q.get()
>
> assert myNewVar == myVar
> print len(myNewVar), "bytes piped around."
>
>
> --
> --Bryan
>


Great, it works, thank you Bryan !

Could you explain me why you use a queue instead of a simple array for
getting the piped var ?

Regards,

Ben

 
Reply With Quote
 
 
 
 
Bryan Olson
Guest
Posts: n/a
 
      03-21-2008
Benjamin Watine wrote:
> (E-Mail Removed) a écrit :
>> I wrote:
>>> And here's a thread example, based on Benjamin's code:

>> [...]
>>
>> Doh! Race condition. Make that:
>>
>> import subprocess
>> import thread
>> import Queue
>>
>> def readtoq(pipe, q):
>> q.put(pipe.read())
>>
>> cat = subprocess.Popen('cat', shell=True, stdin=subprocess.PIPE,
>> stdout=subprocess.PIPE)
>>
>> myVar = str(range(1000000)) # arbitrary test data.
>>
>> q = Queue.Queue()
>> thread.start_new_thread(readtoq, (cat.stdout, q))
>> cat.stdin.write(myVar)
>> cat.stdin.close()
>> cat.wait()
>> myNewVar = q.get()
>>
>> assert myNewVar == myVar
>> print len(myNewVar), "bytes piped around."


>
> Great, it works, thank you Bryan !
>
> Could you explain me why you use a queue instead of a simple array for
> getting the piped var ?


The call to q.get() will block until an item is in the queue.
At that point in the program, we had already waited for cat to
terminate:

cat.wait()
myNewVar = q.get()

But passing cat.wait() does not imply that our own thread has
already read all of cat's output and put it in some destination
object. Data could still be in transit.

My first version, subsequently titled, "Doh! Race condition,"
worked in all of several runs of its built-in test. Doesn't
make it right.


--
--Bryan
 
Reply With Quote
 
 
 
 
Benjamin Watine
Guest
Posts: n/a
 
      03-25-2008
Bryan Olson a écrit :
> Benjamin Watine wrote:
>> (E-Mail Removed) a écrit :
>>> I wrote:
>>>> And here's a thread example, based on Benjamin's code:
>>> [...]
>>>
>>> Doh! Race condition. Make that:
>>>
>>> import subprocess
>>> import thread
>>> import Queue
>>>
>>> def readtoq(pipe, q):
>>> q.put(pipe.read())
>>>
>>> cat = subprocess.Popen('cat', shell=True, stdin=subprocess.PIPE,
>>> stdout=subprocess.PIPE)
>>>
>>> myVar = str(range(1000000)) # arbitrary test data.
>>>
>>> q = Queue.Queue()
>>> thread.start_new_thread(readtoq, (cat.stdout, q))
>>> cat.stdin.write(myVar)
>>> cat.stdin.close()
>>> cat.wait()
>>> myNewVar = q.get()
>>>
>>> assert myNewVar == myVar
>>> print len(myNewVar), "bytes piped around."

>
>> Great, it works, thank you Bryan !
>>
>> Could you explain me why you use a queue instead of a simple array for
>> getting the piped var ?

>
> The call to q.get() will block until an item is in the queue.
> At that point in the program, we had already waited for cat to
> terminate:
>
> cat.wait()
> myNewVar = q.get()
>
> But passing cat.wait() does not imply that our own thread has
> already read all of cat's output and put it in some destination
> object. Data could still be in transit.
>
> My first version, subsequently titled, "Doh! Race condition,"
> worked in all of several runs of its built-in test. Doesn't
> make it right.
>

OK, so if I understand well what you said, using queue allow to be sure
that the data is passed in totality before coninuing with next
instruction. That make sense.
Using thread and queue seems to be very more slow than using files
redirection with bash. I'll see if it make a great load average and/or
I/O time.

Thanks again for your help Bryan.

Ben
 
Reply With Quote
 
Bryan Olson
Guest
Posts: n/a
 
      03-26-2008
Benjamin Watine wrote:
> OK, so if I understand well what you said, using queue allow to be sure
> that the data is passed in totality before coninuing with next
> instruction. That make sense.


Right.

> Using thread and queue seems to be very more slow than using files
> redirection with bash. I'll see if it make a great load average and/or
> I/O time.


Hmmm... you might try increasing the buffer size.


--
--Bryan

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
Difference between Session["var"] and Session("var") thomson ASP .Net 10 06-20-2005 01:02 PM
Difference between Session["var"] and Session("var") thomson ASP .Net 0 06-20-2005 10:54 AM
Threads.. Session var lost, App var ok Alvin Bruney ASP .Net 1 12-02-2003 01:56 AM
does "struct_name var = { 0 }; " fill var with 0? Fred C++ 3 08-10-2003 09:44 AM



Advertisments