get http

Discussion in 'NZ Computing' started by Shane, Jul 28, 2005.

  1. Shane

    Shane Guest

    Ive written me a little script in perl that logs into a website, follows a
    link, and then reads the info of the final page, (w00t! ;-)
    The only thing is, getting the whole page of html seems to me to be a
    waste of bandwidth, and I only want a few pieces of info from the page.
    Is there a way to get a webserver to send me only the relevant parts of a
    page (either through perl, or some other means, I can convert later?)

    TIA


    --
    Hardware, n.: The parts of a computer system that can be kicked

    The best way to get the right answer on usenet is to post the wrong one.
    Shane, Jul 28, 2005
    #1
    1. Advertising

  2. In article <-a-geek.net>,
    Shane <-a-geek.net> wrote:

    >Ive written me a little script in perl that logs into a website, follows a
    >link, and then reads the info of the final page, (w00t! ;-)
    >The only thing is, getting the whole page of html seems to me to be a
    >waste of bandwidth, and I only want a few pieces of info from the page.
    >Is there a way to get a webserver to send me only the relevant parts of a
    >page (either through perl, or some other means, I can convert later?)


    There's an accept-byterange option somewhere, which is used by the
    Acrobat Reader plugin if available. It probably won't be of much use for
    HTML, for the simple reason that there's nothing in the initial part of
    an HTML file that gives any indication of offsets to other interesting
    parts. (Adobe had to bring out a new update of PDF specifically to add
    this kind of information.)
    Lawrence D’Oliveiro, Jul 28, 2005
    #2
    1. Advertising

  3. Shane

    Harry Guest

    Shane wrote:

    > Ive written me a little script in perl that logs into a website, follows a
    > link, and then reads the info of the final page, (w00t! ;-)
    > The only thing is, getting the whole page of html seems to me to be a
    > waste of bandwidth, and I only want a few pieces of info from the page.
    > Is there a way to get a webserver to send me only the relevant parts of a
    > page (either through perl, or some other means, I can convert later?)
    >


    You can specify byte ranges but that presumes that you know
    the byte range you want. You can also have the page returned only
    if it has changed since some date. And if the format of the page
    remains much the same (like fixed format fields only are
    changing - stock prices etc) then you might be able to use byte ranges.

    You can also try and force compressed replies but I don't know
    of any http server that obeys the rfc and actually compresses anything.
    Pity because the traffic on the web could drop by over 50% if they did.
    Harry, Jul 28, 2005
    #3
  4. Shane

    Shane Guest


    >>

    > You can specify byte ranges but that presumes that you know the byte range
    > you want. You can also have the page returned only if it has changed
    > since some date. And if the format of the page remains much the same (like
    > fixed format fields only are changing - stock prices etc) then you might
    > be able to use byte ranges.
    >
    > You can also try and force compressed replies but I don't know of any http
    > server that obeys the rfc and actually compresses anything. Pity because
    > the traffic on the web could drop by over 50% if they did.


    Ive discovered the pm I am using supports If-modified-since, but requires
    a cached page, which I will get with a Accept: text/html request
    (hopefully) to limit bandwidth, then if the webserver allows me, I will
    use the if-modified-since header to check on updates which is what I am
    really after
    After that I will code it into VB, and sell the .exe for a bazillion
    dollars (or post the code in perl so everyone can point and laugh :-(

    --
    Hardware, n.: The parts of a computer system that can be kicked

    The best way to get the right answer on usenet is to post the wrong one.
    Shane, Jul 31, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jog Dial
    Replies:
    1
    Views:
    566
    Russell Lusignan
    Aug 26, 2004
  2. a.nonny mouse
    Replies:
    2
    Views:
    1,094
  3. Scott
    Replies:
    1
    Views:
    8,870
    ScottF
    Aug 4, 2004
  4. Jake72
    Replies:
    24
    Views:
    857
    [ Doc Jeff ]
    Jun 29, 2004
  5. milan_9211

    HTTP SOAP/HTTP GET/HTTP POST

    milan_9211, Jan 10, 2011, in forum: Software
    Replies:
    0
    Views:
    3,084
    milan_9211
    Jan 10, 2011
Loading...

Share This Page