Click Official ELI Links
Get Help With Your Extortion Letter | ELI Phone Support | ELI Legal Representation Program
Show your support of the ELI website & ELI Forums through a PayPal Contribution. Thank you for supporting the ongoing fight and reporting of Extortion Settlement Demand Letters.

Author Topic: Is there a definitive IP range for PICSCOUT - i want to kill the spider!  (Read 46620 times)

Robert Plant

  • Newbie
  • *
  • Posts: 6
    • View Profile
Re: Is there a definitive IP range for PICSCOUT - i want to kill the spider!
« Reply #15 on: February 01, 2013, 06:17:53 AM »
For your typical small non-retail business, would you really want anything cataloguing your site besides google?

jot

  • Jr. Member
  • **
  • Posts: 25
    • View Profile
Re: Is there a definitive IP range for PICSCOUT - i want to kill the spider!
« Reply #16 on: February 01, 2013, 07:35:57 AM »
I digress... would blocking these IPs in .htaccess prevent them from impacting server resources?  I assume they would completely ignore robots.txt.
If blocked in .htaccess, blocking in robots.txt becomes superfluous. However, if you are blocking by IP and you miss an IP block or a robot changes IP ranges, it won't work. robots.txt might-- if the bot obeys it (which it may not.)

It was Getty Images.  I am already blocking a good range of IP addresses and two domain names with Picscout.

lucia

  • Hero Member
  • *****
  • Posts: 767
    • View Profile
Re: Is there a definitive IP range for PICSCOUT - i want to kill the spider!
« Reply #17 on: February 01, 2013, 05:36:57 PM »
Depend what's "typical".  You might want yahoo search, bing and a few others. If you are a blogger you might want incoming pings from blogs. You might  want feed readers visiting.  If you sell advertizing, there might be a few bots that are worth letting visit. (I don't know what they are, but they may exist.)  Many people like the wayback. (Some don't.)

But really, there are a stupendous number of things visiting. I have no idea why *anyone* outside China would want baidu spider to visit. Similar for yandex but with Russia. I don't know why anyone who is not retail wants a "shopping bot" (the kind that find good prices on retail items for people to compare) to visit. 

So, I can't say "only google" categorically. But I'd say if you pick a 'mystery' bot that visits a lot at random, chances it does you no good exceed 90%.

ws2001

  • Newbie
  • *
  • Posts: 14
    • View Profile
Here is what I have currently on Pic-Scout and others.  I have not updated my file in a while so others may have additional info.

One of our sites was pounded (mini DDOS) through multiple IPs. The IPs lead to Bezeq International-Ltd, and information on them lead me/us here.

- Hello - Wave -

Thought the following might help those who want to block Getty Image's rogue BI bots.

Bezeq International-Ltd
http://www.nirsoft.net/countryip/il.html  <-- no connection to them  8)
31.168.0.0-31.168.255.255
62.219.0.0-62.219.255.255
79.176.0.0-79.183.255.255
81.218.0.0-81.218-255.255
82.80.0.0-82.81.255.255
84.108.0.0-84.111.255.255
85.130.128.0-85.130.255.255
109.64.0.0-109.67.255.255
212.5.64.0-212.25.127.255
212.179.0.0-212.179.255.255
217.22.112.0-217.22.127.255

We were hit by bots from three of the above blocks, so it appears ok. But you should double-check. Especially since we're new here.

Best of luck.

brianjclark

  • Newbie
  • *
  • Posts: 19
    • View Profile
There is a good bit on this forum about using your htaccess and basically saying to the bots "don't go to these areas please". If they ignore your warning you ban them!

http://forums.eukhost.com/newreply.php?do=newreply&p=87709

Just applied it to all my sites plus the little addition of emailing me when one is banned. Within 20 seconds of putting it in place, blinkin Googlebot came along, completely disregarded the robots.txt file and got itself banned. Well there ya go :(

Robert Krausankas (BuddhaPi)

  • ELI Defense Team Member
  • Administrator
  • Hero Member
  • *****
  • Posts: 3354
    • View Profile
    • ExtortionLetterInfo
There is a good bit on this forum about using your htaccess and basically saying to the bots "don't go to these areas please". If they ignore your warning you ban them!

http://forums.eukhost.com/newreply.php?do=newreply&p=87709

Just applied it to all my sites plus the little addition of emailing me when one is banned. Within 20 seconds of putting it in place, blinkin Googlebot came along, completely disregarded the robots.txt file and got itself banned. Well there ya go :(
[/quote

Googlebot DOES adhere to robots.txt, chances are good this bot was simply masking itself as googlebot , If it were me, I would be doing some further digging into this...IP address, ect....
Most questions have already been addressed in the forums, get yourself educated before making decisions.

Any advice is strictly that, and anything I may state is based on my opinions, and observations.
Robert Krausankas

I have a few friends around here..

brianjclark

  • Newbie
  • *
  • Posts: 19
    • View Profile
Well it did resolve back to Google, but maybe they were going by what the robot.txt said a few hours ago. I can understand they might want to cache it etc.

"The IP 66.249.75.237 (crawl-66-249-75-237.googlebot.com) has been blocked for an invalid access attempt to a file, directory, or a scanning attempt."

I think I can let them off maybe once or twice and take them back out of my block list ;)

brianjclark

  • Newbie
  • *
  • Posts: 19
    • View Profile
Over time there seems to be a gradual trickle of bots poking about. Some tell the truth and say who they are while others quite clearly try to cover their tracks and make out they are Mozilla browsers. I have looked up the IPs to some of there and they resolve to things that quite clearly are not browsers, as there are no ways to link to the "booby traps" on my sites with a normal browser.

Remains an interesting subject!

## Banned IPs
Deny from 220.181.108.158
# baiduspider-220-181-108-158.crawl.baidu.com

Deny from 94.242.198.110
# Agent: Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko Firefox/11.0)
# on 2013-07-03 (Wed) 02:38:11 IP: 94.242.198.110 (static-198-110.softronics.ch)

Deny from 27.45.240.84
# Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729))
# on 2013-07-03 (Wed) 06:33:30 IP: 27.45.240.84 (27.45.240.84) <-- China Unicom Guangdong Province Network

Deny from 27.45.240.82
# Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729))
# on 2013-07-07 (Sun) 12:45:13 IP: 27.45.240.82 (27.45.240.82) <-- China Unicom Guangdong Province Network

Deny from 175.42.90.137
# Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729))
# on 2013-07-09 (Tue) 06:29:28 IP: 175.42.90.137 (175.42.90.137) <-- China Unicom Fujian Province Network

Deny from 76.94.95.83
# Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0)
# on 2013-07-11 (Thu) 18:52:09 IP: 76.94.95.83 (cpe-76-94-95-83.socal.res.rr.com) <-- Road Runner / Time Warner Cable

Deny from 183.234.49.109
# Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729))
# on 2013-07-13 (Sat) 09:59:59 IP: 183.234.49.109 (183.234.49.109) <-- China Mobile communications corporation

Deny from 14.211.88.3
# Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729))
# on 2013-07-13 (Sat) 14:32:40 IP: 14.211.88.3 (14.211.88.3) <-- CHINANET Guangdong province network

Deny from 50.19.165.99
# Agent: Test Spider 0.2)
# on 2013-07-14 (Sun) 01:25:24 IP: 50.19.165.99 (ec2-50-19-165-99.compute-1.amazonaws.com)

Deny from 188.143.234.127
# Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1))
# on 2013-07-08 (Mon) 13:29:20 IP: 188.143.234.127 (188.143.234.127) <-- ToussaintDesaulniers-net

Deny from 192.114.71.13
# Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:11.0) Gecko/20100101 Firefox/11.0)
# on 2013-07-10 (Wed) 17:40:51 IP: 192.114.71.13 (bzq-114-71-13.static.bezeqint.net) <-- Bastards

Deny from 89.75.96.207
# Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.94 Safari/537.36)
# on 2013-07-11 (Thu) 08:39:09 IP: 89.75.96.207 (89-75-96-207.dynamic.chello.pl) <-- PL-UPC-20060222 in Warsaw

Deny from 5.10.83.73
# Agent: Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/))
# on 2013-07-14 (Sun) 05:41:12 IP: 5.10.83.73 (5.10.83.73-static.reverse.softlayer.com) <-- Ahrefs Pte Ltd Singapore

Deny from 188.143.234.127
# Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1))
# on 2013-07-08 (Mon) 13:29:20 IP: 188.143.234.127 (188.143.234.127)<-- ToussaintDesaulniers-net

Deny from 192.114.71.13
# Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:11.0) Gecko/20100101 Firefox/11.0)
# on 2013-07-10 (Wed) 17:40:51 IP: 192.114.71.13 (bzq-114-71-13.static.bezeqint.net)  <-- Bastards

Deny from 89.75.96.207
# Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.94 Safari/537.36)
# on 2013-07-11 (Thu) 08:39:09 IP: 89.75.96.207 (89-75-96-207.dynamic.chello.pl) <-- PL-UPC-20060222 in Warsaw

Deny from 5.10.83.73
# Agent: Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/))
# on 2013-07-14 (Sun) 05:41:12 IP: 5.10.83.73 (5.10.83.73-static.reverse.softlayer.com) <-- Ahrefs Pte Ltd Singapore

Deny from 37.59.202.77
# Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1)
# on 2013-07-07 (Sun) 14:08:04 IP: 37.59.202.77 (37.59.202.77) <-- Str Miron Costin, Brasov, France

Deny from 207.189.121.44
# Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100721 Firefox/3.6.8)
# on 2013-07-08 (Mon) 01:31:17 IP: 207.189.121.44 (207.189.121.44) <-- VIAWEST-NETBLOCK-207.189.96.0/19

Deny from 5.135.47.74
# Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0b7pre) Gecko/20100921 Firefox/4.0b7pre)
# on 2013-07-08 (Mon) 11:00:01 IP: 5.135.47.74 (5.135.47.74) <-- 2 rue Kellermann, France

--- end ---

brianjclark

  • Newbie
  • *
  • Posts: 19
    • View Profile
The Bezeq International IPs now appear to start "192."   

That's a nice touch to anyone thinking that's ur local internal network (192.168 etc etc.)

ws2001

  • Newbie
  • *
  • Posts: 14
    • View Profile
For Bezeq International add 192.114.64.0-192.114.79.255

Looks like some cough lawyer is reading the posts and passing the news of exposed IPs.

Robert Krausankas (BuddhaPi)

  • ELI Defense Team Member
  • Administrator
  • Hero Member
  • *****
  • Posts: 3354
    • View Profile
    • ExtortionLetterInfo
I knew it was only a matter of time, before new IP's started appearing...hence I simply block Israel as a whole..at the server firewall and the on the server.
Most questions have already been addressed in the forums, get yourself educated before making decisions.

Any advice is strictly that, and anything I may state is based on my opinions, and observations.
Robert Krausankas

I have a few friends around here..

jbigfoot

  • Jr. Member
  • **
  • Posts: 32
    • View Profile


TinEye.com

You may also wish to exclude TinEye.com. TinEye is a program like pic Scout that crawls the web taking samples of images off of page webpages. It then stores these images and you can go to the website and upload an image and it will show you all other instances where it has found this image on the Internet. Getty has also been known to use TinEye as a quick and easy method of locating webpages in which to send demand letters to.

I've been using TinEye, on the recommendation of ELI, to check and see if images I have or others are using are public domain or copyrighted. What are the liabilities for me?

Robert Krausankas (BuddhaPi)

  • ELI Defense Team Member
  • Administrator
  • Hero Member
  • *****
  • Posts: 3354
    • View Profile
    • ExtortionLetterInfo
there are no "liabilities for you using tineye to research images...whoever can you explain how tineye is telling you whether images are public domain or copyrighted?

FYI: Copyright exists at the moment of creation, so a good majority of images are copyrighted..
Most questions have already been addressed in the forums, get yourself educated before making decisions.

Any advice is strictly that, and anything I may state is based on my opinions, and observations.
Robert Krausankas

I have a few friends around here..

jbigfoot

  • Jr. Member
  • **
  • Posts: 32
    • View Profile
can you explain how tineye is telling you whether images are public domain or copyrighted?

I didn't explain myself fully on this one.

The images in question have been on my web site for years, as a tribute to my father. They are pictures of the Distinguished Flying Cross and the Air Medal. I didn't remember when or where I got them, so I used TinEye to see if these pictures appeared anywhere else on the web. They did; and the links that TinEye came back with were federal government web sites (if I recall), and these web sites clearly state that there is no copyright restriction for the use of these images.

Robert Krausankas (BuddhaPi)

  • ELI Defense Team Member
  • Administrator
  • Hero Member
  • *****
  • Posts: 3354
    • View Profile
    • ExtortionLetterInfo
can you explain how tineye is telling you whether images are public domain or copyrighted?

I didn't explain myself fully on this one.

The images in question have been on my web site for years, as a tribute to my father. They are pictures of the Distinguished Flying Cross and the Air Medal. I didn't remember when or where I got them, so I used TinEye to see if these pictures appeared anywhere else on the web. They did; and the links that TinEye came back with were federal government web sites (if I recall), and these web sites clearly state that there is no copyright restriction for the use of these images.

Thanks for the clarification! Getty has been known to accuse folks of infringement over public domain images.. If the images on those government sites you mention are indeed the same, you could sternly point this out to getty, and advise them if they continue their harassment, you will file complaints with the Washington Attorney General, and you will also invoice them for the time you have been wasting on this matter.
Most questions have already been addressed in the forums, get yourself educated before making decisions.

Any advice is strictly that, and anything I may state is based on my opinions, and observations.
Robert Krausankas

I have a few friends around here..

 

Official ELI Help Options
Get Help With Your Extortion Letter | ELI Phone Support Call | ELI Defense Letter Program
Show your support of the ELI website & ELI Forums through a PayPal Contribution. Thank you for supporting the ongoing fight and reporting of Extortion Settlement Demand Letters.