Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > gevent + urllib2 blocking

Reply
Thread Tools

gevent + urllib2 blocking

 
 
月忧茗
Guest
Posts: n/a
 
      02-22-2013
Hi,

One problem, thanks for help.


import gevent.monkey
gevent.monkey.match_all()

from lxml import etree
# I using xpath parse the html


def _get(p):
url = BUILD_URL(p)
html = urllib2.urlopen(url)

# RUN AT HERE AND BLOCKING

# ver1
tree = etree.parse(html, parse)

# ver2
# tree = etree.fromstring(html.read(), parse)

....


workers = [gevent.spawn(_get, i) for i in xrange(1, 100)]
gevent.joinall(workers)



When using ver1, etree.parse(html, parse)
I noticed there are only two greenlets runs at "RUN AT HERE AND BLOCKING"

others are not running!


But, when I using ver2, this code will running normally as expected.


Why this ?

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
cunfused why gevent block redis' socket request? Tony Shao Python 0 12-29-2012 03:46 AM
concurrent.futures vs gevent subin Python 0 07-29-2012 06:37 PM
why greenlet, gevent or the stackless are needed? self.python Python 4 07-07-2012 03:38 PM
Problem with: urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) Josef Cihal Python 0 09-05-2005 11:26 AM
Blocking and non blocking assignment in VHDL Hendra Gunawan VHDL 1 04-08-2004 06:03 AM



Advertisments