IP and internal links (poss search engine or spyware bot).

Discussion in 'Computer Support' started by Billy, Jul 21, 2005.

  1. Billy

    Billy Guest

    Hi,

    I was looking through a website I own, and I noticed the following in
    the logs. The thing is that the website is undeveloped and I didn't log
    on at that time or even look at it. The index.html is just as the host
    left it with no changes and the URLs seem to refer to internal web
    pages. Is this a concern? Thanks.

    #Date: 2005-07-20 20:47:39
    #Fields: date time c-ip cs-username s-sitename s-computername s-ip
    s-port cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status
    sc-bytes cs-bytes time-taken cs-version cs(User-Agent) cs(Cookie)
    cs(Referer)
    2005-07-20 20:47:39 THEIR IP - W3SVC1448 APP506 SERVER'S IP 80 GET / -
    200 0 12803 181 968 HTTP/1.1 Mozilla/4.0 (compatible; MSIE 6.0; Windows
    XP) - -
    2005-07-20 20:48:03 THEIR IP - W3SVC1448 APP506 SERVER'S IP 80 GET
    /cp/scripts/index.html - 200 0 6241 255 390 HTTP/1.1 Mozilla/4.0
    (compatible; MSIE 6.0; Windows XP) - http://www.mywebsite.org/
    2005-07-20 20:48:31 THEIR IP - W3SVC1448 APP506 SERVER'S IP 80 GET
    /cp/scripts/ASP/index.html - 200 0 783 280 2578 HTTP/1.1 Mozilla/4.0
    (compatible; MSIE 6.0; Windows XP) -
    http://www.mywebsite.org/cp/scripts/index.html
    2005-07-20 20:48:32 THEIR IP - W3SVC1448 APP506 SERVER'S IP 80 GET
    /cp/scripts/Perl/index.html - 200 0 783 281 922 HTTP/1.1 Mozilla/4.0
    (compatible; MSIE 6.0; Windows XP) -
    http://www.mywebsite.org/cp/scripts/index.html
    2005-07-20 20:48:33 THEIR IP- W3SVC1448 APP506 SERVER'S IP 80 GET
    /cp/scripts/PHP/index.html - 200 64 0 280 312 HTTP/1.1 Mozilla/4.0
    (compatible; MSIE 6.0; Windows XP) -
    http://www.mywebsite.org/cp/scripts/index.html

    And so on for a few more links. Total links visited = 16. Total time
    spent = 1 minute and 22 seconds.

    I assume this might be a search engine but if so, how can it access
    internal links?

    A tracertert search times out...although their IP is 63.148.99.239 which
    a google search reveals that other users want to block it. I guess my
    main question is how does the search bot have access to the internal links?

    How would a robots.txt work to deal with this?
     
    Billy, Jul 21, 2005
    #1
    1. Advertisements

  2. Billy

    Duane Arnold Guest

    Duane Arnold, Jul 21, 2005
    #2
    1. Advertisements

  3. Billy

    pcbutts1 Guest

    Format and reinstall.
    You have spyware.
    Just for the weekend.
    Do you think that is a concern, Billy?
    Thanks for sharing that with us.
    http://www.mywebsite.org/cp/scripts/index.html

    How did a little girl like you land a great big job like this?
    So everyone can become enlightened.
    Moderation in everything.
    And you ask me why I'm happy.
    You have to keep refreshing.
    Who wants to block it?
    I have no idea. How does the search bot have access to the internal links?
    How do you like that? You're full of shit.
     
    pcbutts1, Jul 21, 2005
    #3
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.