Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Controlling source IP address within urllib2

Reply
Thread Tools

Controlling source IP address within urllib2

 
 
Dan
Guest
Posts: n/a
 
      06-03-2005
Does anybody know how to control the source IP address (IPv4) when
using the urllib2 library? I have a Linux box with several IP
addresses in the same subnet, and I want to simulate several
individuals within that subnet accessing web pages independently. I
need the functionality of urllib2 because there will be redirects and
other HTTP-type functions to implement. It would be nice if I could
create (and bind) sockets myself and then tell the urllib functions to
use those sockets. Perhaps there is some sort of "back-door" way of
doing this??? Any hints are appreciated!
-Dan

 
Reply With Quote
 
 
 
 
John J. Lee
Guest
Posts: n/a
 
      06-04-2005
"Dan" <(E-Mail Removed)> writes:

> Does anybody know how to control the source IP address (IPv4) when
> using the urllib2 library? I have a Linux box with several IP
> addresses in the same subnet, and I want to simulate several
> individuals within that subnet accessing web pages independently. I
> need the functionality of urllib2 because there will be redirects and
> other HTTP-type functions to implement. It would be nice if I could
> create (and bind) sockets myself and then tell the urllib functions to
> use those sockets. Perhaps there is some sort of "back-door" way of
> doing this??? Any hints are appreciated!


There's no built-in support, but I expect it's very easy to do. I
suppose the easiest way would be to derive from
httplib.HTTPConnection, overriding .connect() to call .bind() on the
socket. Call it BindingHTTPConnection, then define:

class BindingHTTPHandler(urllib2.HTTPHandler):
def http_open(self, req):
return self.do_open(BindingHTTPConnection, req)


Same goes for https:

if hasattr(httplib, 'HTTPS'):
class BindingHTTPSHandler(urllib2.HTTPSHandler):
def https_open(self, req):
return self.do_open(BindingHTTPSConnection, req)


Personally, I probably wouldn't use build_opener(), but since the
above classes derive from the corresponding urllib2 classes, you can
do:

opener = urllib2.build_opener(BindingHTTPHandler, BindingHTTPSHandler)
response = opener.open('http://www.example.com/')


John
 
Reply With Quote
 
 
 
 
Dan
Guest
Posts: n/a
 
      06-06-2005
John,
Thanks for your input. I can kind of see the light in this, but I'm
having difficulty knowing where the "do_open" method comes from. Also,
I'll need to follow redirects, so I assume then I would add a
HTTPRedirectHandler instance to the urllib2.build_opener. (?) Thanks
again for your help.
-Dan

 
Reply With Quote
 
John J. Lee
Guest
Posts: n/a
 
      06-07-2005
"Dan" <(E-Mail Removed)> writes:

> John,
> Thanks for your input. I can kind of see the light in this, but I'm
> having difficulty knowing where the "do_open" method comes from. Also,


AbstractHTTPHandler


> I'll need to follow redirects, so I assume then I would add a
> HTTPRedirectHandler instance to the urllib2.build_opener. (?) Thanks


No. It does that by default.


John
 
Reply With Quote
 
Dan
Guest
Posts: n/a
 
      06-08-2005
John,
Thanks again for your help!
I think that the do_open function in AbstractHTTPHandler does not
return the correct object type as required by the opener. When I
include the code you recommended, the implementation comes back with
the message, "urlopen error unknown url type: http". Strange, because
I would think that overriding the "http_open" function in the handler
would have signaled that this function is capable of handling http. If
I call the HTTPHandler base class "http_open" function from within the
derived class, all works okay, but of course, I don't get to use the
source IP address I wanted to use.
I'll keep trying and let you know what I find.
-Dan

 
Reply With Quote
 
Dan
Guest
Posts: n/a
 
      06-10-2005
Just FYI ... I finally got this to work. Unfortunately, I was unable
to use the urllib2 library. The embedded Linux team here informed me
that the urllib2 library will not be available on the machine I wanted
to deploy on, so I had to go back to using the urllib and httplib
libraries. After trying several things, I finally gave up and hacked
the library code directly. It turned out to be quite easy. I just had
to wedge in a "self.sock.bind()" call in front of the
"self.sock.connect()" call within the HTTPConnection class inside the
httplib library. (I used the bind() function to nail down the source
IP address of the client.) Of course, I had to weave in some code that
allowed me to pass the client IP address through the URLopener class in
the urllib library. Everything seems to work so far.

John, thanks again for your help. You pointed me in the right
direction.

-Dan

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Controlling the bindings within an eval Adam C. Javascript 4 07-10-2008 04:56 PM
Controlling different HTML forms with 1 button within vb.net =?Utf-8?B?RGFuaWVs?= ASP .Net 1 05-10-2007 04:41 PM
Problem with: urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) Josef Cihal Python 0 09-05-2005 11:26 AM
controlling an iframe from another iframe within a frame James Wallace Javascript 1 05-14-2004 05:18 PM
controlling redirection within frames headware ASP .Net 0 08-04-2003 11:46 PM



Advertisments