Do you know if any Clients check out this Forum? Do they even know we exist here?
Shopping Central Jersey Shoreline. WHAT? I'm an adult?! When did this happen?! How do I make it stop?!
@ wrote:
User-agent: *
Disallow: /
@ wrote:
#####################################################
##Blocking based on User-Agent Header
SetEnvIfNoCase ^User-Agent$ .*(Googlebot|msnbot|AltaVista|Slurp|Yahoo|craftbot|download|extract|stripper|sucker|ninja|clshttp|spider|leacher|collector|grabber|webpictures) HTTP_SAFE_BADBOT
SetEnvIfNoCase ^User-Agent$ .*(libwww-perl|aesop_com_spiderman) HTTP_SAFE_BADBOT
Deny from env=HTTP_SAFE_BADBOT
#####################################################
## Blocking with RewriteCond
RewriteCond %{HTTP_USER_AGENT} ^.*(Googlebot|msnbot|AltaVista|Slurp|Yahoo|craftbot|download|extract|stripper|sucker|ninja|clshttp|spider|leacher|collector|grabber|webpictures).*$ [NC]
RewriteRule . - [F,L]
@89lulalula89 wrote:
Wouldn't it be easier to just provide excellent customer service?
@scanman1 wrote:
This will keep out the bad search engines that don't listen to the robots.txt file that is an agreed way to not place the site in the search engine.