Problems with CloudFlare
By Strictly-SoftwareRecently I moved a couple of my sites behind the free Cloudflare proxy option and set the DNS so it pointed to their server rather than the 123reg.co.uk ones I had been using.
However before I did this I tested out whether there was much difference between the sites.
1. I set to use Cloudflare and WP Super Cache.
2. I used WP Super Cache, Widget Cache and WP Minify.
I actually found that that the 2nd set-up gave me better results. Why I don't know.
However in the end due to all the spam and apparent blocking Cloudflare claimed to be able to do I set the other site up behind it.
However after a while I noticed a few things you might want to be aware of
1. I had a number of email scripts that sent out thousands of emails and I used a
set_time_limit(5000);
at the top of the file to ensure it didn't time out by the Virtual Servers standard 30 seconds.
Also and in-between each email (which I appended to a log file) I did a wait command with:
sleep(2);
So that I didn't overkill the server.
However you should be aware that when you use set_time_limit and then use functions like sleep or file_put_contents or file_get_contents the time it takes to wait, access files and retrieve data is not included in the time limit.
Also as I was using Cloudflare, my PHP script which is web based so I can easily call it by hand if I need to, was using the standard domain e.g http://www.mysite.com/mysendemailjob.php.
However Cloudflare it seems has it's own timeout limit which overrides anything you set in Apache or in your PHP file of about 60 seconds.
I noticed this because my script kept bombing out after around 60 seconds and returned a Cloudflare 524 error which you can read about here.
So to get round this problem I used the "direct" domain method they set as default to bypass CloudFlare.
I have obviously changed mine as you should to for security sake but once changed to something else when I ran the url: http://www.mysite.com/mysendemailjob.php it didn't bomb out any-more and carried on until the end.
Another thing you have to be careful about CloudFlare is if you have spent ages filling your IP Table up with IP addresses that you want banned due to spam, hack attempts or just over use.
Because all IP addresses in the Apache Access Log are now CloudFlare addresses these won't be used and you are now relying on ClouldFlare's own security measures to block dangerous IP's.
The same goes for your .htaccess file. If you have banned a whole countries range say China or Russia (biggest hackers on the earth - apart from the NSA of course) then these ranges won't mean jack as the user from China will be going through a CloudFlare proxy IP address to your site so any IP you had banned him from will now be useless.
The only thing left to do is ban by user-agent, blank agents is a good one and so are short ones (less than 10 characters as they are usually jibberish).
I ban most of the standard HTTP libraries like CURL, WGET, WIN HTTP, Snoopy and so on as most script kiddies download a library, and don't even bother changing the user-agent before crawling and spamming. Therefore if someone isn't going to tell me who they really are then they can get a 403!
So they are a few things to watch out for with Cloudflare.
I know you can get modules that replace the Cloudflare IP with the original users IP but if you are on an old Debian Lenny box then they don't have support for that.
They must be supplying x-forwarded-for or other headers as when I did a scan using the bypass URL I got back my original IP but with a www.mysite.com scan I got back CloudFlare IPS e.g 104.28.25.11 etc.
The only thing you can do if you cannot take a modern module and reverse engineer it to older code is use the WordPress CloudFlare plugin that lets you get real Akismet IP addresses so you can still ban them.
It is a pain and one I am debating on whether to return to the days of before Cloudflare where my own security measures meant I banned over 50% of traffic and my server bills didn't go up and up.
"CloudFlare is supposed to save me bandwidth but ever since I have installed it although it claims it has saved me loads my Rackspace bill for bandwidth use has gone up and up!"
So just be careful when using Cloudflare it may seem like a magic tool but all those rocketscripts they add to your code are just "async" attributes and you can get many a plugin to minify your source and compress it on the server without using PHP to do so.
The choice is yours but be warned!
Posts
4 comments:
For your IP banning: CloudFlare sends along the user's original IP in a header. Just grab the contents of the header "CF_CONNECTING_IP".
Yes they do as well as passing it along in the X-Forwarded-For header.
However tell me in your .htaccess file how you ban a load of IP addresses with the
DENY,ALLOW
Format of IP addresses.
The only thing I can think of is grabbing the header out of a RewriteCond rule and then some regex with a list of IPs in it.
As for the firewall, they have basically told me it is ignored and replaced by their Threat Firewall. I actually proved this to them by trying to ban an IP that kept coming through with a 200 status code.
Only the threat panel or a PHP header script with an array of IP's that could be checked and returned a 403 status worked.
Solutions welcome.
Hi,
We hosted our application on public network using cloudflare for some pages we are getting 524 error. In our local network those pages are working fine. We are using asp.net application. Please help us to resolve that issue.
I can't solve another companies problem with a proxy in front of your site you will need to talk to cloudflares help team or write a ticket, they do answer them.
However as it's a timeout error I would suggest looking at your Apache config files as it means it is trying to connect to your site but you are not responding in time. So you may have too many visitors trying to visit your site at one time (check your CPU, Memory, Disk Swap) in the console with a TOP command.
Also check your Apache file is not set to have 254 connections at one time (or whatever number it is) as that means 254 people/BOTS will try and visit your site at one time and if it isn't super fast and cached up for them then they will all wait in a queue driving up your CPU.
It is better to have a lower number like 12 so that each connection may wait a little time before the page loads but at least will load unlike now.
I have a BASH script which you should run on your console and give you an appropriate number if it's really low like 2 then I suggest you need more memory or you are disk swapping too much. If you are not on a dedicated server then another site could be causing the issue.
Check out >http://blog.strictly-software.com/2013/07/apache-performance-tuning-bash-script.html
Also this WordPress plugin of mine might help as it will email you key stats about your memory/CPU if it cannot load a page > https://www.strictly-software.com/plugins/strictly-system-check (more at https://www.strictly-software.com/plugins/) that plugin is very useful for me as it also analyses Cloudflare status codes as well as 500,400,300 etc.
Have a read of the article and run the bash script and check your Apache config file to make sure it is not default as too many "connections allowed" at one time will crash your site. Read up also on Twitter Rushes, I don't know if you Tweet but if you do the TweetBOT plugin on https://www.strictly-software.com/plugins/strictly-tweetbot might help you get your page cached before any tweets are sent out.
But I have my connection at 16 so people may have to wait but so what the page loads - I also ban social media BOTS and crawlers from places I would never be interested in e.g China, India, Mongolia as that is just wasted bandwidth you are paying for.
Check it all out.
Post a Comment