Posts tagged site

As I was browsing my lighttpd logs (something fun to pass time), I noticed a few weird entries. 62% of the visitors to my site use Firefox, so seeing quite a few IE visitors set off some alarm bells.

::ffff:208.80.193.40 nick125.com - [22/Sep/2009:07:47:52 -0600] "GET / HTTP/1.0" 200 15810 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; MEGAUPLOAD 1.0; ZangoToolbar 4.8.3; SpamBlockerUtility 4.8.4; MSN 9.0;MSN 9.1; MSNbQ002; MSNmen-us; MSNcOTH)"::ffff:208.80.193.27 continental.nick125.com - [22/Sep/2009:07:50:16 -0600] "GET / HTTP/1.0" 200 3569 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; PeoplePal 3.0; ZangoToolbar 4.8.3)"


..and so on.

Odd. I decide to whois the IP….and guess who it is!

OrgName: Websense, Inc
OrgID: WEBSE-6
Address: 10240 Sorrento Valley Road
City: San Diego
StateProv: CA
PostalCode: 92121
Country: US

Hmm. Why are they spoofing their user-agent? Also, it appears that they ignore robots.txt, as they never tried to request it from my server:

nick@lithium:~$ grep -i '208.80.19[2-9]' /var/log/lighttpd/access.log | grep -i 'robots'
nick@lithium:~$

Not only are they spoofing their UA, but they’re also disregarding robots.txt! How nice of them.

Anyone else notice this?