Terms of Service Prohibiting Crawlers and Bots?

[JLorimer]

I apologize if this topic has already been discussed in this capacity.  I tried to search for something similar but it either hasn't been or I didn't try the right combination of keywords.

Since I have been thrown into this whole mess I have been kind of interested in how legally stable it would be to add something to a site's terms of service stating that crawler and bot activity is prohibited.  Say for example that I put the correct meta tags, robots.txt, .htaccess, and additional scripts in place that (at least attempt to) completely block crawler activity.  I would then state in my terms of use that I have made a reasonable and faithful effort to prohibit all types of bots and crawlers and that any such traffic is a violation of my terms and an intrusion into my site. 

As many of us know, crawlers and such can eat up 50% or greater of our system resources.  In periods of high traffic, that literally means that legitimate users cannot access the site.  Some might even call that denial of service.

My thinking is also that if PicScout or something else would find an image and they would attempt to sue, you could question the means by which the image was discovered.  If it was discovered that a banned tool was used, could you then show the terms of service, the methods used to block such a tool, and prove that the means of discovery is considered to be a system intrusion?  I certainly don't desire that type of traffic on any of my sites.  I only allow Google and Bing because they actually bring positive traffic to my site in most cases.  Even in cases where all images are legal I would not want to waste my bandwidth on such a tool that is only seeking to catch me in potential wrongdoing. 

To me it seems like if police were randomly walking down the street and letting themselves into your house to browse around just because you have a standard door and windows on your dwelling.  Sure, you open your house to people who are welcome but you make efforts to lock doors and windows and keep most other people out. 

I realize this would be an extremely gray area because they you might be able to potentially define all kind of other strange things in your terms.  It just seems that something to that effect would be reasonable since there are valid concerns about bandwidth usage.  The fact that you can ask Google, Bing, The Internet Archive, and any other respectable bot not to crawl your site and they will obey your wishes seems that it would also be reasonable. 

Thoughts?

[stinger]

I am not a lawyer, but my thought is that "breaking and entering" is handled in a different court than the one that handles digital image copyright issues.

You might be right, but you might have to take someone to court to prove that you are right.  Maybe someone with more legal experience can set the record straight here.

[lucia]

My thinking is also that if PicScout or something else would find an image and they would attempt to sue, you could question the means by which the image was discovered.  If it was discovered that a banned tool was used, could you then show the terms of service, the methods used to block such a tool, and prove that the means of discovery is considered to be a system intrusion? 

No one knows the answer to that. But Craigslist and 3Taps are duking it out in court.
http://www.volokh.com/2013/08/18/district-court-holds-that-intentionally-circumventing-ip-address-block-is-unauthorized-access-under-the-cfaa/
Note that it's unlikely you have sent any cease and desists to Picscout or other bots. So you haven't worked as hard as Craigslist has at keeping 3Taps out.

Note that Picscout violating cfaa would not necessarily affect your copyright dispute. It just means you might sue them for violating cfaa, and they might sue you for copyright.

I realize this would be an extremely gray area because they you might be able to potentially define all kind of other strange things in your terms.  It just seems that something to that effect would be reasonable since there are valid concerns about bandwidth usage.  The fact that you can ask Google, Bing, The Internet Archive, and any other respectable bot not to crawl your site and they will obey your wishes seems that it would also be reasonable. 

Thoughts?

My thoughts are it's a legal gray area. Huge gray. But so far: unless you have specifically instructed Picscout to keep out and figured out how to ban their IP, no judge will decree they violate CFAA by visiting.  Even if the judge did, that probably wouldn't affect any copyright violation finding.

[JLorimer]

That's a good read.  I had not heard of that dispute before.

Note that it's unlikely you have sent any cease and desists to Picscout or other bots. So you haven't worked as hard as Craigslist has at keeping 3Taps out.

That's true.  It would be interesting to see Craiglist come out on top of this one.  I say that only because I find it terrible that a troll can scan your site, disregard DMCA process (as they have in many cases by not sending a proper takedown) and go right to a lawsuit.  But if you would attempt to keep them from wasting your bandwidth and system resources then it would require a cease and desist letter to every bot owner, who would then need to violate a second time.  It seems like an equation that is increasingly hard to balance in your own favor, let alone equally.

Note that Picscout violating cfaa would not necessarily affect your copyright dispute. It just means you might sue them for violating cfaa, and they might sue you for copyright.

That makes sense.   I was thinking along the lines of illegally obtained evidence being thrown out.  Maybe that only happens on TV.

Part of me was hoping it would create scenarios where you could take the owner of the crawler to court and they would say "our machine didn't read your terms of service" and you could respond like they do by saying it doesn't matter if you read it or not because it is still there and legal.

My thoughts are it's a legal gray area. Huge gray. But so far: unless you have specifically instructed Picscout to keep out and figured out how to ban their IP, no judge will decree they violate CFAA by visiting.  Even if the judge did, that probably wouldn't affect any copyright violation finding.

It seems a big part of it would also be having enough money to throw away in the process of fighting in court.  That's money I don't have.  I would love to see somebody try something like it though.

[lucia]

Do we know details of any case that has gone to trial?  If so, is there any mention of what means were used in the end to prove the infringement?  I have attempted to look at various scanned documents.  I admittedly understand very little of what I read in legal documents.

I don't. There's a Getty case. These details didn't matter in that case because the judge ruled that Getty's copyrights were flawed. Consequently, they lacked the right to sue.  How Getty found the infringement didn't matter because they couldn't even file. 

But seriously, I don't think we need to look at any cases to assess the likelihood that you can use robots.txt or TOS to prevent someone from filing or winning a copyright suit. There is nothing in the copyright statute that says the way the infringement came to light matters in any way shape of form.  It's how evidence came to light could matter in a criminal case-- in that situation the 4th amendment applies. But the 4th amendment restricts evidence from the Government and its agents from introducing evidence obtained through illegal searches in trials. Getty is not the government; I'm pretty sure the 4th amendment is irrelevant here.
 
If you were to try that tack, which you are free to do, you'd be trying to bring up something entirely new. It's really much better to avoid using getty images rather than hoping some TOS would help in the event that you do accidentally infringe and the image is found by Picscout.

[JLorimer]

I think you quoted me from another thread.  These are really separate thoughts even though I'm on here all the time because of what I'm currently going through.  I wouldn't want to try to introduce that at all.  The chance of losing seems that it would multiply significantly.

This thread is here because the more I have thought about the amount of my system resources that may actually have become to devoted to tools like picscout trying to catch me for something, the more upset it makes me all the time.  I mentioned elsewhere that I have literally had traffic problems caused by crawlers, bots, and spammers overrunning my site.  Legitimate users were honestly blocked because of these scummy tools.

[lucia]

This thread is here because the more I have thought about the amount of my system resources that may actually have become to devoted to tools like picscout trying to catch me for something, the more upset it makes me all the time.  I mentioned elsewhere that I have literally had traffic problems caused by crawlers, bots, and spammers overrunning my site.  Legitimate users were honestly blocked because of these scummy tools.

I'm totally with you on that one. If you use php resources, I recommend ZBblock for those.  (It can't protect static resources.)

[Robert Krausankas (BuddhaPi)]

You also need to consider the fact that Picscout is out of Israel...US laws means nothing to them, anything that is not password protected is fair game, and would be considered "public" IMO. BOTS/Crawlers are not going to read your terms of use, you would need to have something that is clicked and agreed to by the user, this would probably stop the bots, but also be a huge turn off to real users..

[JLorimer]

That brings me to the question: how binding are any website terms of use then?  For example, most sites have something that says a little bit about browsing the site means you agree to the terms.  And by agreeing to the terms by simply browsing, you often agree that images are not fair game, text cannot be copied, etc.  I realize there are other laws at play, but isn't it kind of the same thing?  I mean, just because your bot didn't read or agree to my terms doesn't mean they aren't there and being enforced.

[Robert Krausankas (BuddhaPi)]

True..I tend to look at it like a contract..kinda..with a click through,  it could serve as a digital signature of sorts. without it, it's just a page that most users don't read anyway, which makes it difficult at best to enforce. Hell even Getty's contributor agreement was deemed useless by a judge, the contributor signed on, with a digital signature, but no-one at Getty "signed" anything digital or otherwise...when asked about it, they kinda just shrugged and "didn't know".. Look at any of the big sites out there, in order to register, sign-up, ect..you have to click something agreeing to those terms..without it your SOL for the most part.

[ws2001]

As noted earlier, this is Getty's take ('do as I say, not as I do'):
http://imagery.gettyimages.com/gettylive/dm/gbr/TermsConditions_pc.html
"You are specifically prohibited from: (a) downloading, copying, or re-transmitting any or all of the Site or the Getty Images Content without, or in violation of, a written licence or agreement with Getty Images; (b) using any data mining, robots or similar data gathering or extraction methods;"

"Such unauthorized use may also violate applicable laws including without limitation copyright and trademark laws, the laws of privacy and publicity, and applicable communications regulations and statutes."

Put similar text in your .htaccess and robots.txt files. Won't stop bots, but at least you've done your 'due diligence'.

Best to block them:
http://www.extortionletterinfo.com/forum/getty-images-letter-forum/is-there-a-definitive-ip-range-for-picscout-i-want-to-kill-the-spider!/
http://www.extortionletterinfo.com/forum/getty-images-letter-forum/probable-picscout-or-image-scanner/
http://www.extortionletterinfo.com/forum/getty-images-letter-forum/new-picscout-ip-range-to-block