«

How can I obtain the source of a remote webpage (e.g.
http://www.python.org/index.html) using Python?

Something like:

pyPage = open('http://www.python.org/index.html',r).read()

Obviously that won't work but how can I do something to that effect?
Thanks,
Ryan Kaskel

--I posed this before but it seems it is not showing up...

(Ryan Kaskel) writes:
> Something like:
>
> pyPage = open('http://www.python.org/index.html',r).read()
>
> Obviously that won't work but how can I do something to that effect?

import urllib
pyPage = urllib.urlopen('http://www.python.org/index.html',r).read()

Paul Rubin <http://> writes:
> import urllib
> pyPage = urllib.urlopen('http://www.python.org/index.html',r).read()

oops:

import urllib
pyPage = urllib.urlopen('http://www.python.org/index.html').read()

i.e. omit the 'r' argument to urlib.urlopen.

> pyPage = open('http://www.python.org/index.html',r).read()

using open() for local files and ORLs is called url-fopen and works in
PHP, which is a major security hole, because it even allows one to
include() code files from the web without knowing it, that kind of thing...

python has two functions so you know what you're doing.

If your webpage needs cookies or something, you'll need urllib2

If you wanna parse it afterwards use Htmllib or BeautifulSoup

Take a look at the urllib module:

http://python.org/doc/2.3.3/lib/module-urllib.html

On Wed, Jun 23, 2004 at 10:03:04PM -0700, Ryan Kaskel wrote:
> How can I obtain the source of a remote webpage (e.g.
> http://www.python.org/index.html) using Python?
>
> Something like:
>
> pyPage = open('http://www.python.org/index.html',r).read()
>
> Obviously that won't work but how can I do something to that effect?
> Thanks,
> Ryan Kaskel
>
> --I posed this before but it seems it is not showing up...