Beautiful BeautifulSoup

Discussion in 'NZ Computing' started by Lawrence D'Oliveiro, Jan 4, 2009.

  1. BeautifulSoup is a Python library for scraping websites. It will even try to
    make sense of malformed HTML. Here's how easy it is to grab strips off the
    dilbert.com site:

    ----
    import sys
    import re
    import urllib
    from BeautifulSoup import \
    BeautifulSoup as SoupParse

    url_base = "http://dilbert.com"

    TempFilename, Headers = urllib.urlretrieve("%s/strips/?Page=%d" % (url_base, int(sys.argv[1])))
    Document = SoupParse(open(TempFilename, "r").read())
    for img in Document.findAll("img", src = re.compile(r"\.strip\.gif$")) :
    filename = re.search(r"([^\/]+)$", img["src"]).group(1)
    img_url = "%s/%s" % (url_base, img["src"])
    sys.stderr.write("%s => %s\n" % (img_url, filename)) # debug
    urllib.urlretrieve(img_url, filename)
    #end for
    ----

    For instance, the oldest strips are currently on page 691. So the call

    get_dilbert 691

    will download the strips on that page into the current directory.
    Lawrence D'Oliveiro, Jan 4, 2009
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Baron Wolfgar von Schtuldeworfshiseundurheimhoppen

    A beautiful story

    Baron Wolfgar von Schtuldeworfshiseundurheimhoppen, Oct 6, 2003, in forum: Computer Support
    Replies:
    17
    Views:
    772
    Olivers
    Dec 1, 2003
  2. Idris Alooma
    Replies:
    5
    Views:
    575
    Petit Alexi
    Oct 9, 2003
  3. Boomer

    Beautiful kittens

    Boomer, Feb 14, 2004, in forum: Computer Support
    Replies:
    28
    Views:
    1,142
  4. R2D2

    Beautiful babe

    R2D2, Dec 18, 2003, in forum: Digital Photography
    Replies:
    23
    Views:
    2,832
    Guest
    Dec 24, 2003
  5. Rune Austefjord

    Beautiful printout results

    Rune Austefjord, Dec 24, 2003, in forum: Digital Photography
    Replies:
    5
    Views:
    315
    Annika1980
    Dec 25, 2003
Loading...

Share This Page