Saturday 3 March 2012

The Wordpress Survival Guide Part 3 - Security

This is the 3rd part of the Wordpress Survival Guide which looks at security measures.

The other two guides which cover basics for people new to Linux, Apache and Wordpress and Performance can be found here:

The Wordpress Survival Guide Part 1 - Linux, Apache and Wordpress
The Wordpress Survival Guide Part 2 - Performance

If you have an under powered or busy server then security and performance go hand in hand as reducing the amount of traffic from bad bots, hackbots, spammers, login hackers, heavy hitters and so on will also help reduce the load on your server.

There are many plugins out there which claim to help the security on Wordpress but you should be careful as from my own investigation of the code many of these plugins whilst protecting you from potential threats can reduce your sites performance as they carry out too many checks on submitted fields.

If a plugin is checking every form element submitted to the server for hundreds of known SQL injection or XSS hacks with regular expressions or string checks then this can slow down a page load incredibly.

Therefore the further up the chain you can push your security checks from PHP code running in Wordpress to the actual web server the better.

The aim is to move as much blocking code away from your site to your server so we want to make use of our firewall and our .htaccess file by adding a number of rules designed to identify and block potential hackers and spammers before they get to your site and any plugin code.

Blocking with our LINUX Firewall

Once you have found persistent offenders from the methods listed below the aim is to remove any CPU  and Memory from being wasted on them by WordPress and your .htaccess file and put them into your LINUX Firewall.

You can install a plugin to your server called Fail2Ban which will actually analyse your log files for you looking for spammers, hackers and bandwidth wasters and add them automatically to your IPTables (Network Firewall).

However you should read up on it carefully and configure it correctly so that you don't end up blocking yourself sending emails into WordPress or other actions.

The higher up you can block the bad traffic the better. Therefore read this article on how you can block bad BOTS and users by the WebMin interface.

Blocking with the .htaccess file

The .htaccess file sits in your websites root folder and contains rules local to the site which can allow or deny users to your site by blocking certain requests either by IP address, user-agent, or the type of request the user is making. 

I used to return a 403 forbidden status code to the people I wanted to block but I am now trying out a different format which seems to have increased performance. I suspect this might be down to the users of malicious bots seeing a 403 Forbidden code as a "challenge" to crack rather than a sign they should go away therefore I have replaced returning 403 with a 404 code.

As there doesn't seem to be a quick flag like [F] for 403 to use you should create your own 404 page which should contain very basic HTML and no references to any Wordpress include files or other code that could be loaded in.

At the top of you page you put some PHP to return the 404 status code. An example is below.

header("Status: 404 Not Found");
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "">
<html xmlns="" dir="ltr" lang="en-US">
<head profile="">
<title>My Website</title>
<link rel="canonical" href="" />
<body id="home">
 <div id="page-wrap">
  <div id="header">  
   <p id="slogan">This is my website</p>
 <div id="body">
  <h1>My Website</h1>

  <p>Sorry that page doesn't exist</p>

The idea is to have a quick loading page that returns a basic response rather than a blank one so that crawlers think they have just made a mistake and that the URL they are targeting doesn't exist. A blank response or a forbidden status could signify to them that you have caught them out. The PHP at the top ensures a 404 status code is returned.

Once you have created the custom 404.php page and put some basic text in it upload it to the root of your website.

Now you can edit your .htaccess file and change some of the main checks we are going to do so that they redirect the bot to the custom 404.php page and not the Wordpress 404 page.

We don't want to get ourselves in a big loop of circular redirects which is why we check for the 404.php page on our 2nd block of rules.

The first set of rules block common SQL injection attacks, common XSS hacks which include passing JavaScript in the querystring, known file lookups as well as calls to certain applications which should never be accessible from the webserver but sometimes are.

The second block is aimed at known bad bot user-agents, common HTTP libraries such as CURL, WGet, Snoopy and other libraries which are usually downloaded by Script Kiddies and used without any modification.

A proper hacker or spammer will mask themselves a lot better than this but these rules will stop the wannabes and baby nobs that have no clue what they are doing but still overload your server.

I also then block blank and very short user-agents or jbberish user-agents as I believe if the user cannot tell me who they are then I don't want them on my site. It is up to you whether you decide you want people masking themselves in this way to access your servers. You will notice that on this section I still use the [F] forbidden flag and return a 403 code.

The last block are known email harvesters and spammers which I redirect off to a honeypot to be logged and blocked by a proper tool designed to catch out email harvesting bots.

I have found that a good set of rules can reduce traffic to a server by over 50% which is obviously a major performance benefit and since I have changed my first two sets of rules from returning 403 to 404 codes the response time of my server and sites upon it has increased.

<IfModule mod_rewrite.c>
Options +FollowSymlinks
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{QUERY_STRING} (%3C|<)/?script(%3E|>)    [NC,OR]
RewriteCond %{QUERY_STRING} (eval\(|document\.|\.cookie|createElement)    [NC,OR]
RewriteCond %{QUERY_STRING} DECLARE[^a-z]+\@\w+[^a-z]+N?VARCHAR\((?:\d{1,4}|max)\)    [NC,OR]
RewriteCond %{QUERY_STRING} ^/+\?.*sys.?(?:objects|columns|tables|[xs]p_|exec)    [NC,OR]
RewriteCond %{REQUEST_URI} ^\/\/?(owssvr|strmver|Auth_data|redirect\.adp|MSOffice|DCShop|msadc|winnt|system32|script|autoexec|formmail\.pl|_mem_bin|NULL\.) [NC,OR]
RewriteCond %{REQUEST_URI} ^\/\/?(php\-?my\-?admin\-?\d?|P\/?M\/?A(\d+)?|(db|web)?(admin|db|sql)|(my)?sql\-?(admin|manager|web)?)/? [NC]
RewriteRule ^.*$ /404.php [R=301,L]

RewriteCond %{REQUEST_FILENAME} !/404\.php # ensure we are not already on our 404.php page
RewriteCond %{HTTP_USER_AGENT} (?:ColdFusion|Jakarta|HTTPClient|Java|libwww\-perl|Nutch|PycURL|Python|Snoopy|urllib) [NC,OR] # common HTTP libraries
RewriteCond %{HTTP_USER_AGENT} (?:LWP|PECL|POE|PycURL|WinHttp|curl|Wget) [OR] # case sensitive HTTP libraries
RewriteCond %{HTTP_USER_AGENT} (?:ati2qs|cz32ts|EventMachine|indy|linkcheck|Morfeus|NV32ts|Pangolin|Paros|ripper|scanner|offline) [NC,OR] # known rippers
RewriteCond %{HTTP_USER_AGENT} (?:AcoiRobot|alligator|auto|bandit|boardreader|BCD2000|blackwidow|capture|ChinaClaw|collector|copier|disco|devil|downloader|fetch|flickbot|grabber|gosospider|Gentoo|HTMLParser|hapax|hook|igetter|jetcar|JS-Kit|kame-rt|kmbot|KKman|leach|majestic|MetaURI|mole|miner|mirror|mxbot|rogerbot|race|reaper|sauger|speedy|Sogou|sucker|snake|spinn3r|Sosospider|stripper|UnwindFetchor|vampire|whacker|xenu|zeus|zip) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (?:AhrefsBot|fairshare|proxy|PageGetter|magpie|Zemanta|baidu|MiniRedir|SurveyBot|PMAFind|SolomonoBot|whitehat|blackhat|MSIE\s6\.0|ZmEu) [NC]
RewriteRule ^.*$ /404.php [R=301,L]

# Block blank or very short user-agents. If they cannot be bothered to tell me who they are or provide jibberish then they are not welcome!                                          
RewriteCond %{HTTP_USER_AGENT} ^(?:-?|[a-z1-9\-\_]{1,10})$ [NC]
RewriteRule .* - [F,L]

# fake referrers and known email harvesters which I send off to a honeytrap full of fake emails
RewriteCond %{HTTP_USER_AGENT} (?:atomic|collect|e?mail|magnet|reaper|siphon|sweeper|harvest|(?:microsoft\surl\scontrol)|wolf) [NC,OR] # spambots, email harvesters
RewriteCond %{HTTP_REFERER} ^[^?]*(?:iaea|\.ideography|addresses)(?:(\.co\.uk)|\.org\|.com) [NC]
RewriteRule ^.*$ [R,L] 


You should check your websites access and error logs regularly to see who has been getting banned or 404'd a lot by these rules and then you can decide whether to block their IP address by adding it your .htaccess file like so.

order allow,deny
deny from # example IP - not known to be bad

These rules deny all requests from a particular IP address.

OR adding it to your Firewall if it's making a large amount of calls to your site.

One trick I do like, which I have to thank a commenter, RobinInTexas for is this rule which sends the BOT back to the IP address they came from in the first place. However there are two changes to the rule he sent which used

http://%{REMOTE_ADDR} [L,R=301]

And that is to return them to their localhost address NOT the IP they came from as many people will be going through gateways such as people at work, phone or tablet users and people on WIFI systems in shops etc.

You don't want the ISP that this gateway belongs to thinking you are sending it lots of hackers and bad bots as you might get your site blocked for sending so much traffic to it.

The other change is to make it a 302 temporary redirect instead of a 301 as it is the correct status code to use. So instead of the rule above use this. [L,R=302]

You could decide to send them to a honeypot website that logs them as a bad BOT so other users know about them or even some sites that are designed to keep them crawling for days wasting time going through links that lead nowhere but to other links that lead nowhere etc.

The rule in action would look like this:

RewriteCond %{HTTP_USER_AGENT} (?:Spider|MJ12bot|seomax|atomic|collect|e?mail|magnet|reaper|tools\.ua\.random|siphon|sweeper|harvest|(?:microsoft\surl\scontrol)|wolf) [NC]
RewriteRule .* [L,R=302]

2. Blocking SSH Attacks with DenyHosts

Install DenyHosts if you havent already which will block attacks by SSH to your server. It is amazing how many people I have blocked since installing this application and people are always on the look out for new webservers on known cloud hosting IP ranges like Rackspace to attack and hopefully compromise.

To install this you open an SSH connection (with Putty) and run the following commands.

apt-get install denyhosts

to view by console go to the directory the application is installed to.

cd /var/log/denyhosts
tail -f /var/logs/denyhosts

This will show you the tail end of the DenyHosts log file and any newly added IP addresses.

Make sure to add any IP addresses that you access your server console by SSH to the Allow Hosts which you can do by the terminal in VI or from WebMin by going to:

Webmin > Networking > TCP Wrappers > AllowedHosts > Add New Rule

Fill out the form like so:

Services: ALL
Remote hosts: Tick the second radio button and add the IP address to the text input
Shell Commands: none
Save the form.

3. Stop Being Open To SSH / BASH Hack Attacks

Also to stop your server being vulnerable to hacks like the Shellshock hack which appeared recently and exposed nearly every LINUX machine due to their use of SSH and BASH you should do the following.

Test if you are vulnerable by running this command.

env x='() { :;}; echo vulnerable' dash -c "echo this is a test"

If you are then these are somethings you can do.

Turn off BASH and install DASH an older version.

If you are using DASH and want to run BASH just type in BASH to get there.

Also replace the default shell for root and any other users to another folder with symbolic links. Look up on the web how to do this.

Disable any cgi-bin commands in all Apache config files as this is what the hack relies on e.g

#ScriptAlias /cgi-bin/ /home/searchmysite/cgi-bin/
#allow from all

Remove AW stats and Webalizer for all virtual min sites. These rely on CGI-BIN as well.

Regularly change all your user passwords and especially your root password.

A good technique for a strong password is to thing of a common sentence or phrase you will remember and mix the characters up and add a number on the end only you would remember (not your Birthday!) e.g a football teams last trophy win or the year of your last holiday.

Add some dashed or underscores in as well to make it even harder for password crackers to crack it with dictionary attacks. An example would be.


Regularly check your users table for any that look out of place e.g inserted by a hacker.

Also regularly check your home and temp folders for any files that shouldn't be there. One hack I saw replaced the default SSH config file with a temp file in /tmp/sh that loaded up (using WGET) a file hidden in a website that then ran more WGET commands to load in a library of hacks for DDOS and SSH etc and then ran the commands he wanted.

With a compromised server and a SSH config file that had been overwritten he could then use your server to run hack attacks on other machines.

If this is happening, quickly get the IP of the site he is loading the files from and block incoming and outgoing TCP requests in your firewall. Then get a default SSH config file and replace the hacked version before changing all your passwords and ensuring BASH isn't available to be used in a hack.

You can check if anyone who shouldn't be logged into your machine is with the ps ax command.

4. Using Wordpress Plugins to block dangerous traffic.

Two plugins I have found quite useful so far for reducing hack attacks are these:

The Limit Login Attempts plugin which blocks brute force attacks on the wp-login page. If you don't want people signing up to your site anyway you should use a plugin to obfusicate this page anyway otherwise just limit the number of failed attempts so that dictionary attacks are prevented.

Use the IP addresses this plugin collects and take the worst offenders and put them in your DENY HOSTS table as well as considering banning them with your LINUX firewall. Read this article for more information on banning bad BOTS and blocking hackers and scrapers.

Install the Wordpress Firwewall plugin to block certain hack attempts and be notified by email when attacks occur. Make sure to add any IP address you access your website to the whitelist so you don't get blocked out.

This plugin will look for some of the same tricks our .htaccess file rules are aimed at blocking as well as some different types of attack that are used when form parameters are filled with dangerous values and submitted to the server.

There are other things you can do as well but these 3 tips are a good starting point. I will update this page as and when new features are proven at increasing security without effecting site performance.

4. Using other tools on your server to add rules to DenyHosts and your Firewall

There is a tool you can use on LINUX machines called Fail2Ban which RackSpace and other cloud hosters actually recommend using. It will constantly analyse your access and error logs and add IP addresses which it things are suspicious into your DenyHosts list and your Firewall IP Table.

However be-warned I used it myself and tried some of the email rules. I then found myself having my IP being blocked and emails sent from my own computer to my server blocked.

I then tried removing these from the configuration and still ran into problems (not immediately - so it may not have been Fail2Ban's problem) of emails sent from my PC to my WordPress site where a plugin called Postie put them into the system as articles stopped working.

In the end I had to remove the Fail2Ban program from my server. However if you are not doing anything like I am or can configure it properly (I may have made a mistake) then it could be the tool for you as it will save time adding rules into DenyHosts and your IP TABLE for your firewall to use.

Read Part 1 - An Overview
Read Part 2 - Performance


Unknown said...

Great article. I'm using your htaccess rules on Bartle Doo. More folks should do the same and help clean up the net to at least be the way it was when I started as an Internet Service Provider in 1995.

Thank you.

RobinInTexas said...

Instead of a 403 or 404 which clutters up the server error log I have this:

RewriteCond %{HTTP_USER_AGENT} YYSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Baidu [NC]
RewriteRule .* http://%{REMOTE_ADDR} [L,R=301]

Which quietly sends them where they belong

Rob Reid said...

Yes thats a great idea to send the attackers back to their own site. I like it.

RobinInTexas said...

It's not so much "where" it's the least response you can do short of blocking their it at the server's linux firewall. The server gets a request, and sends a response of about 200 bytes. No error log entry, just "nothing to see here, move along" I don't understand why nearly the whole world insists on creating an access denied error.

Rob Reid said...

Surely htaccess file rules are lower down the chain eg from cloudflare, server firewall, webserver running modules to WP PHP code? So you may not clutter up the error log but the access log will take the hit instead. Just swapping one log file for another and using Apache process power to analyse the request and parse the htaccess file where a Firewall deny doesn't use that CPU?

RobinInTexas said...

I would agree that Cloudflare and the server firewall are far better than htaccess, but anything involving WP is a big drag on the server. I am a big advocate of using Wordfence to cut down unwanted traffic, but WordPress can't even wake up until it talks to the MySQL server. I've done tests that show that depending on WP to enforce www or no www adds about a half second to the response compared to htaccess for those requests that are framed incorrectly.

Rob Reid said...

That's why I wouldn't leave security to WP at all.

The more rings of defence you have the better and the higher up they are - away from the PHP code (1 line = 30 lines of C etc) then the better the performance.

So having as much traffic turned away by Cloudflare or Intrusion detection firewalls the better.

Then let your own servers firewall bat away as much as possible before letting .htaccess do whats needed.

Having some WP security is a good idea as long as it doesn't cost more CPU in detection than the cost of performance in saving you bandwidth and hack attempts.

Things like locking out people hacking your admin account and using CAPTCHAs, BOT Traps and other client side code to trap BOTS that cannot run JS is a good idea.

Having PHP code that analyses every request, form post and so on is overkill and could bring down your server just by having to run 100+ regular expressions to check for known hacks and anonomolies etc.

The further up the security the better - the more you have the better - the more you know about the code you are using the better - especially 3rd party WP security plugins.