Showing posts with label server. Show all posts
Showing posts with label server. Show all posts

Saturday, 24 July 2021

Do Not Be Scared!!!

Main Site And Tools Still Running For Now


By Strictly-Software


Just to let you know that the main site with all the tools such as the:

  • Best JavaScript HTML Encoder & Decoder available, handles double encoding, numeric or entity.
  • Depacking packed code, even if it's been packed multiple times and reformatting it to a readable state.
  • My top WordPress Strictly-AutoTags plugin (Free/Professional), which gives you automated #SEO, deep-linking, and automatic tagging of important words with no 3rd party lookups. No need to check a list of names, if John Smith carries out a wicked crime tonight that is in articles you post automatically then it will find the word and bold it to aid SEO.
  • My top WordPress Strictly-TweetBot plugin, that allows you to tweet to multiple accounts, only if certain words are found in an article OR NOT, using different hash tags made up from either #posttags #categories or #default tags. The professional version has a way of caching the page first before Tweeting so that you are not overwhelmed by any Twitter Rush when you Tweet. Timers can be placed between each Tweet out as well.
  • Lots of free scripts and code that fix holes in frameworks like jQuery.
  • SQL Performance scripts and tuning procedures.
  • Plus lots lots more....
Is still working and you CAN STILL ACCESS IT......

All that has happened is that I can no longer can access the server, due to new security on the site that hosts it. They have added an extra layer of security in which sends an email out to an address I have no access to requesting the code they sent out to be entered on their site to get in. Therefore I cannot get into the servers site, to then use their Java applet to get into the server to add my ever changing IPv4 address to the firewall.

Therefore when you try and go to www.strictly-software.com you will probably see some sort of warning message from your browser like the one shown in the picture below. It will say the site is not secure and hackers maybe trying to steal your data - that is rubbish. 

All that has happened is that due to not being able to renew my SSL certificate, the site is no longer https, no-one is trying to steal your data. I just have not been able to put an SSL on my webserver.

So ignore the message, hit "Advanced", then you should see a "Proceed To Site" link underneath, click that and you can still access the site for now.

Here are two examples, one from Brave/Chrome, the other Opera.

Chrome trying to block access to the site




Opera trying to block access, click the bottom link to get to the site



So no-one is trying to hack you or steal your information, just go to "Advanced" and then click the "Proceed to www.strictly-software.com" link.

All I would suggest is if you are going to buy one of my top WordPress plugins that you do NOT use a PUBLIC WIFI SPOT. 

Do it at home, use a VPN if you can, but do NOT do it at as you are walking down the street being connected and re-connected to various BT public WIFI hotspots

I am sure 99% of them are safe as the users have no technical knowledge to set up their own DNS server and take you somewhere else other than my site, but just to be safe do it at HOME or on a PRIVATE network.

Hope this helps anyone trying to get to the main site.



By Strictly-Software


© 2021 Strictly-Software

Monday, 21 July 2014

Problems with CloudFlare

Problems with CloudFlare

By Strictly-Software

Recently I moved a couple of my sites behind the free Cloudflare proxy option and set the DNS so it pointed to their server rather than the 123reg.co.uk ones I had been using.

However before I did this I tested out whether there was much difference between the sites.

1. I set to use Cloudflare and WP Super Cache.
2. I used WP Super Cache, Widget Cache and WP Minify.

I actually found that that the 2nd set-up gave me better results. Why I don't know.

However in the end due to all the spam and apparent blocking Cloudflare claimed to be able to do I set the other site up behind it.

However after a while I noticed a few things you might want to be aware of

1. I had a number of email scripts that sent out thousands of emails and I used a

set_time_limit(5000);

at the top of the file to ensure it didn't time out by the Virtual Servers standard 30 seconds.

Also and in-between each email (which I appended to a log file) I did a wait command with:

sleep(2);

So that I didn't overkill the server.

However you should be aware that when you use set_time_limit and then use functions like sleep or file_put_contents or file_get_contents the time it takes to wait, access files and retrieve data is not included in the time limit.

Also as I was using Cloudflare, my PHP script which is web based so I can easily call it by hand if I need to, was using the standard domain e.g http://www.mysite.com/mysendemailjob.php.

However Cloudflare it seems has it's own timeout limit which overrides anything you set in Apache or in your PHP file of about 60 seconds.

I noticed this because my script kept bombing out after around 60 seconds and returned a Cloudflare 524 error which you can read about here.


CloudFlare Timeout Error

So to get round this problem I used the "direct" domain method they set as default to bypass CloudFlare.

I have obviously changed mine as you should to for security sake but once changed to something else when I ran the url: http://www.mysite.com/mysendemailjob.php it didn't bomb out any-more and carried on until the end.

Another thing you have to be careful about CloudFlare is if you have spent ages filling your IP Table up with IP addresses that you want banned due to spam, hack attempts or just over use.

Because all IP addresses in the Apache Access Log are now CloudFlare addresses these won't be used and you are now relying on ClouldFlare's own security measures to block dangerous IP's.

The same goes for your .htaccess file. If you have banned a whole countries range say China or Russia (biggest hackers on the earth - apart from the NSA of course) then these ranges won't mean jack as the user from China will be going through a CloudFlare proxy IP address to your site so any IP you had banned him from will now be useless.

The only thing left to do is ban by user-agent, blank agents is a good one and so are short ones (less than 10 characters as they are usually jibberish).

I ban most of the standard HTTP libraries like CURL, WGET, WIN HTTP, Snoopy and so on as most script kiddies download a library, and don't even bother changing the user-agent before crawling and spamming. Therefore if someone isn't going to tell me who they really are then they can get a 403!

So they are a few things to watch out for with Cloudflare.

I know you can get modules that replace the Cloudflare IP with the original users IP but if you are on an old Debian Lenny box then they don't have support for that.

They must be supplying x-forwarded-for or other headers as when I did a scan using the bypass URL I got back my original IP but with a www.mysite.com scan I got back CloudFlare IPS e.g 104.28.25.11 etc.

The only thing you can do if you cannot take a modern module and reverse engineer it to older code is use the WordPress CloudFlare plugin that lets you get real Akismet IP addresses so you can still ban them.

It is a pain and one I am debating on whether to return to the days of before Cloudflare where my own security measures meant I banned over 50% of traffic and my server bills didn't go up and up.

"CloudFlare is supposed to save me bandwidth but ever since I have installed it although it claims it has saved me loads my Rackspace bill for bandwidth use has gone up and up!"

So just be careful when using Cloudflare it may seem like a magic tool but all those rocketscripts they add to your code are just "async" attributes and you can get many a plugin to minify your source and compress it on the server without using PHP to do so.

The choice is yours but be warned!

Tuesday, 10 September 2013

Testing Server Load Before Running Plugin Code On Wordpress

Testing Server Load Before Running Plugin Code On Wordpress

UPDATED - 10th Sep 2013

I have updated this function to handle issues on Windows machines in which the COM object might not be created due to security issues.

If you have a underpowered Linux server and run the bag of shite that is the WordPress CMS system on it then you will have spent ages trying to squeeze every bit of power and performance out of your machine.

You've probably already installed caching plugins at every level from WordPress to the Server and maybe even beyond....into the cloud....all stuff normal websites shouldn't have to do but it seems WordPress / Apache / PHP programmers love doing.

A fast optimised database, queries that return data in sets (not record by record) and some static pages for content that doesn't change constantly should be all you need but it seems that this is not the case in the world of WordPress!

Therefore if you have your own server or virtual server and the right permissions you might want to consider implementing some code in important plugins that prevents the job you intend running causing more performance problems if the server is already over loaded.

You can do this by testing for the current server load, setting a threshold limit and then only running the code you want if the server load is below that limit.

Of course security is key so lock down permissions to your apps and only let admin or the system itself run the code - never a user and never by a querystring that could be hacked!

The code is pretty simple.

It does a split for Windows and non Windows machines and then it checks for a way to test the server load in each branch.

For Windows it has two methods, one for old PHP code and one for PHP 5+.

In the Linux branch it tests for access to the /proc/loadavg file which contains the current load average on LINUX machines.

If it's not there it tries to access the shell_exec function (which may or may not be locked down due to permissions - up to you whether you allow access or not) and if it can run shell commands it calls the "uptime" function to get the current server load from that.

You can then call this function in whatever plugin or function you want and make sure your server isn't already overloaded before running a big job.

I already use it in all my own plugins, the Strictly Google Sitemap and my own version of the WP-O-Matic plugin.


/**
 * Checks the current server load
 *
 * @param boolean $win
 * @return string 
 *
 */
function GetServerLoad(){

 $os = strtolower(PHP_OS); 
 
 // handle non windows machines
 if(substr(PHP_OS, 0, 3) !== 'WIN'){
  if(file_exists("/proc/loadavg")) {    
   $load = file_get_contents("/proc/loadavg"); 
   $load = explode(' ', $load);     
   return $load[0]; 
  }elseif(function_exists("shell_exec")) {     
   $load = @shell_exec("uptime");
   $load = explode(' ', $load);        
   return $load[count($load)-3]; 
  }else { 
   return false; 
  } 
 // handle windows servers
 }else{ 
  if(class_exists("COM")) {     
   $wmi  = new COM("WinMgmts:\\\\."); 
   if(is_object($wmi)){
    $cpus  = $wmi->InstancesOf("Win32_Processor"); 
    $cpuload = 0; 
    $i   = 0;   
    // Old PHP
    if(version_compare('4.50.0', PHP_VERSION) == 1) { 
     // PHP 4      
     while ($cpu = $cpus->Next()) { 
      $cpuload += $cpu->LoadPercentage; 
      $i++; 
     } 
    } else { 
     // PHP 5      
     foreach($cpus as $cpu) { 
      $cpuload += $cpu->LoadPercentage; 
      $i++; 
     } 
    } 
    $cpuload = round($cpuload / $i, 2); 
    return "$cpuload%"; 
   }
  } 
  return false;     
 } 
}

A simple server load testing function that should work across Windows and Linux machines for load testing.

Monday, 27 September 2010

Twitter Traffic is causing high server loads

Wordpress and Twitter Generated Traffic

I came across this interesting article the other day over at cloudtesting.com which listed the user-agents of bots that would visit a link if it was posted on Twitter. I have copied the list below and it's quite amazing to think that as soon as you post a link to your site or blog on Twitter you will suddenly get hammered by X amount of bots.

I can definitely attest to the truthfulness of this behaviour as I am experiencing a similar problem with one of my LAMP Wordpress blogs. Whenever an article is posted I automatically post tweets to 2 (sometimes 3 depending on relevance) Twitter accounts with my new Strictly Tweetbot Wordpress plugin.

Therefore when I import content at scheduled intervals throughout the day I can receive quite a sudden rush of bot traffic to my site which spikes my server load often to levels that are unrecoverable.

  • @hourlypress
  • Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
  • Mozilla/5.0 (compatible; abby/1.0; +http://www.ellerdale.com/crawler.html)
  • Mozilla/5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot
  • Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)
  • Mozilla/5.0 (compatible; Feedtrace-bot/0.2; bot@feedtrace.com)
  • Mozilla/5.0 (compatible; mxbot/1.0; +http://www.chainn.com/mxbot.html)
  • User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)
  • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 OneRiot/1.0 (http://www.oneriot.com)
  • PostRank/2.0 (postrank.com)
  • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 Me.dium/1.0 (http://me.dium.com)
  • Mozilla/5.0 (compatible; VideoSurf_bot +http://www.videosurf.com/bot.html)
  • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
  • Mozilla/5.0 (compatible; page-store) [email:paul at page-store.com]

Personally I think that this list might be out of date as from what I have seen there are quite a few more agents that can be added to that list including bots from Tweetme and Bit.ly.

Currently if I think the bots don't provide any kind of benefit to me in terms of traffic apart from stealing my bandwidth and killing my server I am serving 403's using htaccess rules.

Before banning an agent check your log files or stats to see if you can see any traffic being referred. If you want the benefit but without the constant hitting try contacting the company behind the bot to see if they could change their behaviour. You never know they may be relying on your content and be willing to tweaking their code. We can all live in hope.

Tuesday, 20 July 2010

Strictly System Check Wordpress Plugin

A Site and Server Monitoring Plugin for Wordpress

I have just created a new Wordpress plugin, Strictly System Check, which allows me to easily monitor my Wordpress sites for signs of performance problems such as an overloaded server, too many database connections, corrupt tables or website issues.

I created this plugin primarily for my own use as I noticed that I quite regularly would experience my site going down with the "Error establishing a database connection" error after bulk imports of content after using WP-O-Matic. Whether it was this plugin or another one that caused the problem I don't know but the actual underlying issue is not a misconfigured database but corrupt database tables primarily the wp_options or wp_posts tables.

Therefore the main idea behind the plugin was for it to check my site at scheduled intervals and if it came across this database error message to then scan my MySQL database tables for corrupt tables and if any were found to automatically run a REPAIR statement.

I then extended this functionality to incorporate some more tests including a scan of the HTML source for a known string to ensure that my site is running correctly as well as checks against the webserver to guage the current load average and current database connection count.

The plugin allows webmasters to set their own preferred thresholds so that if the load average was say 2.0 or above then a report would be emailed out to a specified email address informing them of the situation.

Whilst this plugin is not meant to be a replacement for all the great professional site monitoring tools that are available it's a nice add-on that allows website administrators who may not have access to such tools to regularly check the status of their site and to be notified if the site goes down or becomes overloaded.

Please let me know what you think of the plugin and if you like it or it saves your site from going down then please make a donation.

Monday, 8 September 2008

Migrating IIS web application from 32 bit to 64 bit server.

Changing Servers from a 32 to 64 bit environment

By Strictly-Software

I recently had to go through the process of moving a website from a 32 bit windows to 64 bit windows server.

It was one of those tasks that I presumed would be pretty simple and not involve much work but turned into a proper nightmare.

I thought there might have been some sort of “gotcha” guide out on the web somewhere but if there was at the time I couldn’t find it!

I could either find bits and bobs about some individual problems or nothing at all. So I promised myself that on completion I would create an article myself detailing all of the major problems I had to overcome and the solutions I used. So here it is!

The original system was a Windows 2003 server hosting IIS 6.0 and running ASP classic sites.

We had got to the limits in terms of memory allocation and had boosted performance as much as we could in the application (see top 10 tips for boosting ASP classic site performance) therefore we wanted to try moving to a new 64 bit server before considering any large scale application rewrite to .NET.

1. COM objects and compiled dll libraries.

We found that most of our 3rd party COM objects needed upgrading to 64 bit.

You should check each object in turn to see if you need new installations or licences. Objects that we used that needed upgrading included:
  • ASPEmail
  • ASPJPEG
  • ASPUpload
  • DTSearchEngine
  • ISAPI_Rewrite

2. Running scripts.

Scheduled jobs (.bat, .cmd, .vbs) all need to be able to run in a 64 bit environment.

A 64 bit script cannot run a 32 bit process so if your script tries to instantiate 32 bit COM objects
you will get errors unless you upgrade those COM objects to 64 bit (see above).

Another option is to run the script in the c:\windows\sysWOW64\ directory with
cscript which enables you to run 32 bit scripts in a 64 bit environment.

3. ISAPI Rewrite

The name of the rewrite configuration file changes from httpd.ini to .htaccess and if you compare
the format of the files they differ in syntax e.g the flag [I] for "Ignore Case" becomes [NC] "No Case" and a 301 permanent redirect flag of [RP] becomes [R=301].

Also for Rewrite Rules the 64 bit version which is similar to that used on Apache servers requires the first rule to be wrapped in starting and closing tags e.g

# 32 BIT RULE

RewriteRule /some-old-page.htm http://www.newsite.com/newpage.htm [I,O,RP,L]

Becomes

#64 BIT RULE

RewriteRule ^/some-old-page.htm$ http://www.newsite.com/newpage.htm [NC,R=301,L]


There is a helpful conversion tool within IIS that appears in a new tab under each site to convert
your current files to the new format if you don't want to do it by hand.

4. ADO Connection Strings

If you are using an MDAC connection string to connect to an MS SQL database such as:

DRIVER={SQL Server}; SERVER=server_name_or_address;
DATABASE=database_name; UID=username; PWD=password;

You will need to change it to SQLOLEDB as MDAC is not supported on 64 bit windows
systems.

PROVIDER=SQLOLEDB; SERVER=server_name_or_address;
DATABASE=database_name; UID=username; PWD=password;


5. Issues related to changing to an SQLOLEDB connection string.

Changing to the SQLOLEDB provider will mean that you may have problems with SQL that returns multiple recordsets, either client side or server side SQL or stored procedures.

For example with the MDAC connection string, a client side SQL statement like the one below in an example piece of ASP code would work fine.

NOTE the two SELECT statements in the SQL and the objRS.NextRecordset statement against the resulting recordset.


strSQL = "DECLARE @vals varchar(2000); " &_
  "SELECT @vals = COALESCE(@vals + '''', '''', '''''''') + CAST(a.CategoryFK AS varchar(10)) " &_
  "FROM DATA_CATEGORIES_VALUES as a " &_
  "WHERE a.IDFK = 3556 AND " &_
  "a.DataTypeFK = 'JEB';" &_
  "SELECT @vals as val;"

Set objRS = objConnection.Execute(strSQL)

'* move to the next recordset as the first one just builds the string,
'* but it's the 2nd one that returns the values
Set objRS = objRS.NextRecordset

If Not(objRS.BOF AND objRS.EOF) Then
 val = objRS("val")
End If

However using the SQLOLEDB provider will result in an error such as

“Operation is not allowed when the object is closed”

when it tries to move to the next recordset.

I believe that this is caused because within the temporary stored procedures that are created to run client side SQL don't include a “SET NOCOUNT ON” statement which is the first recordset but this line is automatically included by the SQLOLEDB provider and therefore there is no need to move to the next recordset as only one is returned anyway.

Set objRS = objRS.NextRecordset

Using “SET NOCOUNT ON” means that the first recordset implicitly returned by SQL server containing the number of rows affected by the first SELECT statement is not returned.

This would be the case for any SELECT statement that does not itself return a dataset.

6. Stored Procedures and SQLOLEDB

Stored procedures which do not have the SQL statement “SET NOCOUNT ON” at the top of it's code block may also now cause problems if they return multiple recordsets with data and you may get “Operation is not allowed” or “Item does not exist with this name or ordinal” errors when trying to reference the recordset object in your client side code.

Including “SET NOCOUNT ON” at the top of stored procedures is good practise anyway and since SQL 2005 it's part of the default stored procedure template. Therefore any procs that don't include it should be updated to include it to prevent these sort of issues.

7. Returning BLOBS (varchar(max), nvarchar(max), text, ntext etc)

A benefit to changing to the SQLOLEDB provider is that you no longer have to put your BLOBS at the end of your SELECT statements (e.g nvarchar(max), varchar(max), ntext, text).

Previously you had to put these columns at the end of the SELECT statement after any non BLOB columns to prevent them from being returned empty e.g:


SELECT CandidateID, Name, DOB, Town, County, Blob1, Blob2, Blob3


With SQLOLEDB you do not have to do this as it seems to handle them better than MDAC so the following will return values for the BLOB columns even though they are not at the end of the SELECT.


SELECT CandID, Name, Blob1, Blob2, Blob3,  DOB, Town, County

These are the main configuration issues I found when moving to a 64 bit environment but please post any comments if there are other issues related to migrating IIS web applications from 32 to 64 bit platforms.

I am sure there are plenty more!

By Strictly-Software


© 2008 Strictly-Software