You are here

Apache 2, referer spam and .htaccess "deny from" statements

Referer SpamReferer[1] spam seems to be a huge problem nowadays, especially when you'd like to know where your blog visitors are coming from. If you have thousands of fake referrer strings in your logs, you might get the idea that there must be a way to keep the most common spam patterns out of your log files.

Articles like this one suggest to use a .htaccess file with multiple SetEnvIfNoCase statements combined with a final Deny from statement to block common spam patterns.

Such a configuration could look like this:

SetEnvIfNoCase Referer "^http://(www.)?" spam_ref=1

Order Allow,Deny
Allow from all
Deny from env=spam_ref

While this is generally a good idea, one should be really careful when choosing the right place for this configuration.

I ended up doing it the wrong way where I added this at the Apache 2 configuration level so I can have it enabled automatically on all my virtual hosts.

This turned out to be a bad choice as Apache 2 merges configuration directives from configuration and .htaccess files in a way which might actually cause other deny From statements to stop working.

This can open security holes in your applications as some applications place sensible files inside the document root and rely on deny from all statements in their .htaccess files to deny access to those files.

In my case, the deny from statement from the snippet above took precedence over the application's .htaccess directives which in turn made it possible to publicly access otherwise inaccessible files without having to go through the application's authentication layer.

So if you are considering to block referrer spam using the above method, double-check that your applications' security setting are still in effect afterwards.

[1] The misspelled "referer" field is part of the HTTP specification (see RFC 1945).

served by