Wednesday, 20 November 2013

Twitter Changing Their API Again and Parsing Twitter Tweet Responses

Twitter Changing Their API Again and Parsing Twitter Tweet Responses

If you haven't noticed by now you should do the next time you try and use Twitter on the twitter.com website.

Not only are they re-working their API they are cracking down on the sending of links in DM messages.

I personally have had an account suspended within the last week then re-activated 3 days later and I still have had no explanation why this happened from their support team.

Also this happened directly after I was in contact with them about the problems they were having falsely identifying double encoded links as phishing or spam and directing users of my account to a "Twitter thinks this page might be dangerous" page!

Even though their development team claimed the links I was using were okay and safe if you clicked them you would be redirected to this page by mistake.

I think the problem was due to the fact that to get a link into 140 chars you have to first encode it with a link shortener like bit.ly and then Twitter always encode ALL links with their own t.co shortener so they can track the clicks.

I reckon they were having issues identifying the final destination URL and were therefore flagging double encoded links as spam/phishing/hacks etc - despite their claims they weren't.

However now it seems Twitter has changed their DM API. Not only have I found that it constantly refreshes as I am typing causing me to lose my message but if I receive a message as I type it also refreshes causing the same lost wording.

It seems more like a badly written interactive Messenger service now with an AJAX timer to reload new content than the good old DM service it used to be. They still maybe working on it so hopefully these bugs will get fixed soon.

A few people have complained to me they cannot now send links in D or M messages (Direct Messages) and I have found the same problem. It seems Twitter are cracking down on companies that log people who follow and unfollow you as well as treating the majority of links in DM's as spam, whether they are or not.

Anyway bitching time is now over.

What I wanted to write about was that I had to release new versions of my WordPress plugins Strictly AutoTags and Strictly TweetBot the other day.

I was finding on sites with a lot of posts that the Tweets from the TweetBot (which are hooked into the onpublish action/event) were going out before tagging could be completed. Therefore only the default #HashTags were being used.

Therefore if you wanted to use categories or post tags as your #HashTags in the Tweet they were not available at the point of Tweeting.

Therefore I changed the Strictly AutoTags plugin to fire an event once tagging had been completed in the SaveAutoTags method which enabled Strictly TweetBot to run then instead of on publish. I also pass in the Post ID so the event listener knows which post to send Tweets for e.g:


do_action('finished_doing_tagging', $object->ID);

The code in the TweetBot automatically checks for the existence of the AutoTag plugin by looking for a function that I use within the plugin that will only exist if the plugin is active.

If this is found the TweetBot hooks the listener to fire off Tweets into this event instead of the standard on publish event.

As I was explaining to a WordPress developer the other day, I prefer the use of the terms events and listeners due to my event driven coding in JS or C# etc whereas hooks and actions are more WordPress driven.

However as I was re-writing this code I also noticed that in the admin panel of Strictly TweetBots which shows a history of the last Tweets sent and any error messages returned from Twitter that I was not getting back my usual "Duplicate Tweet" error when I tried sending the same tweet multiple times.

By the looks of things the JSON response from Twitter when using OAuth to send a Tweet has changed slightly (when I don't know) and now I have to parse the object and containing array for any error messages rather than checking just for a string as I used to be able to do.

If anyone is interested, or works with OAuth to send Twitter messages, the code to parse the JSON response to collect error messages is below.

Please notice the ShowTweetBotDebug debug function which outputs arrays, objects and strings to the page if needed.


/*
 * My Show Debug Function
 *
 * @param $msg string, object or array
 */
function ShowTweetBotDebug($msg)
{ 
 if(!empty($msg))
 {
  if(is_array($msg)){
   print_r($msg);
   echo "<br />";
  }else if(is_object($msg)){
   var_dump($msg);
   echo "<br />";
  }else if(is_string($msg)){
   echo htmlspecialchars($msg) . "<br>";
  }
 } 
}



$twitterpost = "Some tweet about something";

ShowTweetBotDebug("post this exact tweet = $twitterpost");

// Post our tweet - you should have set up your oauth object before hand using the standard classes
$res = $oauth->post(
 'statuses/update', 
 array(
  'status' => $twitterpost,
  'source' => 'Strictly Tweetbot'
 )
);



ShowTweetBotDebug("response from Twitter is below");

//output errors using vardump to show whole object
ShowTweetBotDebug($res);

// parse error messages from Twitter
if($res){
 // do we have any errors?
 if(isset($res->errors)){
  
  ShowTweetBotDebug("we have errors");

  // if we do then it will be inside an array
  if(is_array($res->errors)){

   ShowTweetBotDebug("we have an array of errors to parse");

   foreach($res->errors as $err){     
    
    ShowTweetBotDebug("Error = " . $err->message);        
   }

  }
 }else{
  ShowTweetBotDebug("tweet sent ok");  

 }
}else{
 ShowTweetBotDebug("Could not obtain a response from Twitter");
}


This code might come in handy to other people who need to parse Twitter responses and just remember - Twitter is ALWAYS changing their API which is a real pain the ass to any developers using their API or trying to make a business model from it.

It seems a lot of companies have had to shut down due to the recent changes to Direct Messages so who knows what they will do in the future?

Wednesday, 23 October 2013

4 simple rules robots won't follow

4 simple rules robots won't follow

Job Rapists and Content Scrapers - how to spot and stop them!

I work with many sites from small blogs to large sites that receive millions of page loads a day. I have to spend a lot of my time checking my traffic log and logger database to investigate hack attempts, heavy hitting bots and content scrappers that take content without asking (on my recruitment sites and jobboards I call this Job Raping and the BOT that does it a Job Rapist).

I banned a large number of these "job rapists" the other week and then had to deal with a number of customers ringing up to ask why we had blocked them. The way I see it (and that really is the only way as it's my responsibility to keep the system free of viruses and hacks) if you are a bot and want to crawl my site you have to do the following steps.

These steps are not uncommon and many sites implement them to reduce bandwidth wasted on bad BOTS as well as protect their sites from spammers and hackers.

4 Rules For BOTS to follow


1. Look at the Robots.txt file and follow the rules

If you don't even bother looking at this file (and I know because I log those that do) then you have broken the most basic rule that all BOTS should follow.

If you can't follow even the most basic rule then you will be given a ban or 403 ASAP.

To see how easy it is to make a BOT that can read and parse a Robots.txt file please read this article (and this is some very basic code I knocked up in an hour or so)

How to write code to parse a Robots.txt file (including the sitemap directive).


2. Identify yourself correctly

Whilst it may not be set in stone, there is a "standard" for BOTS to identify themselves correctly in their user-agents and all proper SERPS and Crawlers will supply a correct user-agent.

If you look at some common ones such as Google or BING or a Twitter BOT we can see a common theme.

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible;bingbot/2.0;+http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; TweetedTimes Bot/1.0; +http://tweetedtimes.com)

They all:
-Provide information on the browser compatibility e.g Mozilla/5.0.
-Provide their name e.g Googlebot, bingbot, TweetedTimes.
-Provide their version e.g 2.1, 2.0, 1.0
-Provide a URL where we can find out information about the BOT and what it does e.g http://www.google.com/bot.htmlhttp://www.bing.com/bingbot.htm and http://tweetedtimes.com

On the systems I control and on many others that use common intrusion detection systems at firewalls and system level (even WordPress plugins). Having a blank user-agent or a short one that doesn't contain a link or email address is enough to get a 403 or ban.

At the very least a BOT should provide some way to let the site owner find out who owns the BOT and what the BOT does.

Having a user-agent of "C4BOT" or "Oodlebot" is just not good enough.

If you are a new crawler identify yourself so that I can search for your URL and see whether I should ban you or not. If you don't identify yourself I will ban you!


3. Set up a Reverse DNS Entry

I am now using the "standard" way of validating crawlers against the IP address they crawl from.

This involves doing a reverse DNS lookup with the IP used by the bot.

If you haven't got this setup then tough luck. If you have then I will do a forward DNS to make sure the IP is registered with the host name.

I think most big crawlers are starting to come on board with this way of doing things now. Plus it is a great way to identify correctly that GoogleBot is really GoogleBot, especially when the use of user-agent switcher tools are so common nowadays.

I also have a lookup table of IP/user-agents for the big crawlers I allow. However if GoogleBot or BING start using new IP addresses that I don't know about the only way I can correctly identify them (especially after experiencing GoogleBOT hacking my site) is by doing this 2 step DNS verification routine.


4. Job Raping / Scraping is not allowed under any circumstances. 

If you are crawling my system then you must have permission from each site owner as well as me to do this.

I have had bots hit tiny weeny itsy bitsy jobboards with only 30 jobs have up to 400,000 page loads a day because of scrapers, email harvesters and bad bots.

This is bandwidth you should not be taking up and if you are a proper job aggregator like Indeed, JobsUK, GoogleBase then you should accept XML feeds of the jobs from the sites who want their jobs to appear on your site.

Having permission from the clients (recruiters/employers) on the site is not good enough as they do not own the content the site owner does. From what I have seen the only job aggregators who crawl rather than accept feeds are those who can't for whatever reason get the jobs the correct way.

I have put automated traffic analysis reports into my systems that let me know at regular intervals which bots are visiting me, which visitors are heavy hitting and which are spoofing, hacking, raping and other forms of content pillaging.

It really is like an arms race from the cold war and I am banning bots every day of the week for breaking these 4 simple to follow rules.

If you are legitimate bot then its not too hard to come up with a user-agent that identifies yourself correctly, set up a reverse DNS entry, follow the robots.txt rules and don't visit my site everyday crawling every single page!

Thursday, 12 September 2013

SEO - Search Engine Optimization

My two cents worth about Search Engine Optimisation - SEO

Originally Posted - 2009
UPDATED - 12th Sep 2013

SEO is big bucks at the moment and it seems to be one of those areas of the web where there seem to be lots of snake oil salesmen and "SEO experts" who will promise no 1 positioning on Google, Bing and Yahoo for $$$ per month.

It is one of those areas that I didn't really pay much attention to when I started web developing mainly because I was not the person paying for the site and relying on leads coming from the web. However as I have worked on more and more sites over the years its become blatantly apparent to me that SEO comes in two forms from a development or sales point of view.

There are the forms of SEO which are basically good web development practise and will come about naturally from having a good site structure, making the site usable and readable as well as helping in terms of accessibility.

Then there are the forms which people will try and bolt onto a site afterwards. Either as an after thought or because an SEO expert has charged the site lots of money, promised the impossible, and wants to use some dubious link-sharing schemes that are believed to work.


Cover the SEO Basics when developing the site


Its a lot harder to just "add some Search Engine Optimization" in once a site has been developed especially if you are developing generic systems that have to work for numerous clients.

I am not an SEO expert and I don't claim to be otherwise I would be charging you lots of money for this advice and making promises that are impossible to be kept. However following these basic tips will only help your sites SEO.

Make sure all links have title tags on them and contain worthy content rather than words like "click here". The content within the anchor tags matter when those bots come a crawling in the dead of night.

You should also make sure all images have ALT attributes on them as well as titles and make sure the content of both differ. As far as I know Googlebot will rate ALT content higher than title content but it cannot hurt to have both.

Make sure you make use of header tags to differentiate out important sections of your site and try to use descriptive wording rather than "Section 1" etc.

Also as I'm sure you have noticed if you have read my blogs before I wrap keywords and keyword rich sentences in strong tags.

I know that Google will also rank emphasised content or content marked as strong over normal content. So as well as helping those readers who skim read to view just the important parts, it also tells Google which words are important on my article.

Write decent content and don't just fill up your pages with visible or non-visible spammy keywords.

In the old days keyword density mattered when ranking content. This was calculated by removing all the noise words and other guff (CSS, JavaScript etc) and then calculating what percentage of the overall page content were relevant keywords? 

Nowadays the bots are a lot cleverer and will penalise content that does this as it looks like spam.

Also its good for your users to have good readable content and you shouldn't remove words between keywords as it makes it more unreadable and you will lose out on the longer 3, 4, 5 word indexable search terms (called long-tail in the SEO world).

Saying this though its always good to remove filler from your pages. For example by putting your CSS and Javascript code into external files when possible and removing large commented out sections of HTML.

You should also aim to put your most important content at the top of the page so its the first thing crawled.

Try moving main menus and other content that can be positioned by CSS to the bottom of the file. This is so that social media sites and other BOTS that take the "first image" on an article and use it in their own social snippets don't accidentally use an advertisers banner instead of your logo or main article picture.

The same thing goes for links. If you have important links but they are in the footer such as links to site-indexes then try getting them higher up the HTML source.

I have seen Google recommend that 100 links a per page is the maximum to have per page. Therefore having a homepage that has your most important links at the bottom of the HTML source but 200+ links above them e.g links to searches even if not all of them are visible then this can be harmful.

If you are using a tabbed interface to switch between tabs of links then the links will still be in the source code and if they are loaded in by JavaScript on demand then that's no good at all as a lot of crawlers don't run JavaScript.

Items such as ISAPI URL rewriting are very good for SEO plus they are nicer URLs for sites to display.

For example using a site I have just worked on as an example http://jobs.professionalpassport.com/companies/perfect-placement-uk-ltd is a much nicer URL to view a particular company profile than the underlying real URL which could also be accessed as http://jobs.professionalpassport.com/jobboard/cands/compview.asp?c=6101

If you can access that page by both links and you don't want to be penalised for duplicate content then you should specify which link you would want to be indexed by specifying your canonical link.You should also use your Robots.txt file to specify that the non re-written URL's are not to be indexed e.g.

Disallow: /jobboard/cands/compview.asp

META tags such as the keywords tag are not considered as important as they once were and having good keyword rich content in the main section of the page is the way to go rather than filling up that META with hundreds of keywords.

The META Description will still be used to help describe your page on search results pages and the META Title tag is very important to describe your page's content to the user and BOT.

However some people are still living in the 90's and seem to think that stuffing their META Keywords with spam is the ultimate SEO trick when in reality that tag is probably ignored by most crawlers nowadays.

Set up a Sitemap straight away containing your sites pages ranked by their importance, how often they change, last modified date etc. The sooner you do this the quicker you site will be getting indexed and gaining site authority. It doesn't matter if it's not 100% ready yet but the sooner it's in the indexes the better.

Whilst you can do this through Googles webmaster tools or Microsofts Bing you don't actually need to use their tools and as long as you use a sitemap directive in your robots.txt file BOTS will find it e.g

Sitemap: http://www.strictly-software.com/sitemap_110908.xml

You can also use tools such as the wonderful SEOBook Toolbar which is an add-on for Firefox which has combined numerous other free online SEO tools into one helpful toolbar. It lets you see your Page Ranking and compare your site to competitors on various keywords across the major search engines.

Also using a text browser such as Lynx to see how your site would look to a crawler such as yahoo or google.is a good trick to see how BOTS would view your site as it will skip all the styling and JavaScript.

There are many other good practises which are basic "musts" in this day and age and the major SERP'S are moving more and more towards social media when it comes to indexing sites and seeing how popular they are.

You should set up a Twitter account and make sure each article is published to it as well as engaging with your followers.

A Facebook Fan page is also a good method of getting people to view snippets of your content and then find your site through the world most popular social media website.

Making your website friendly for people viewing it on tablets or smart phones is also good advice as more and more people are using these devices to view Internet content.

The Other form of SEO, Black Magic Optimization

The other form of Search engine optimization is what I would call "black magic SEO" and it comes in the form of SEO specialists that will charge you lots of money and make impossible claims about getting you to the number one spot in Google for your major keywords and so on.

The problem with SEO is that no-one knows exactly how Google and the others calculate their rankings so no-one can promise anything regarding search engine positioning.

There is Googles Page Ranking which is used in relation to other forms of analysis and it basically means that if you have a site with a high PR that links to your site that does not link back to the original site then it tells Google that your site has higher site authority than the linking site.

If your site only links out to other sites but doesn't have any links coming in from high page ranked relevant sites then you are unlikely to get a high page rank yourself. This is just one of the ways which Google will use to determine how high to place you in the rankings when a search is carried out.

Having lots of links coming in from sites that have nothing whatsoever to do with your site may help drive traffic but will probably not help your PR. Therefore engaging in all these link exchange systems are probably worth jack nipple as unless the content that links to your site is relevant or related in some way its just seen as a link for a links sake i.e spam.

Some "SEO specialists" promote special schemes which have automated 3 way linking between sites enrolled on the scheme.

They know that just having two unrelated sites link to each other basically negates the Page Rank so they try and hide this by having your site A linking to site B which in turn links to site C that then links back to you.

The problem is obviously getting relevant sites linking to you rather than every tom dick and harry.

Also advertising on other sites purely to get indexed links from that site to yours to increase PR may not work due to the fact that most of the large advert management systems output banner adverts using Javascript therefore although the advert will appear on the site and drive traffic when people click it you will not get the benefit of an indexed link. The reason being that when the crawlers come to index the page containing the advert the banner image and any link to your site won't be there.

Anyone who claims that they can get you to the top spot in Google is someone to avoid!

The fact is that Google and the others are constantly changing the way they rank and what they penalise for so something that may seem dubious that works currently could actually harm you down the line.

For example in the old days people would put hidden links on white backgrounds or position them out of site so that the crawlers would hit them but the users wouldn't see which worked for a while until Google and the others cracked down and penalised for it.

Putting any form of content up specifically for a crawler is seen as dubious and you will be penalised for doing it.

Google and BING want to crawl the content that a normal user would see and they have actually been known to mask their own identity ( IP and User-Agent ) when crawling your site so that they can check whether this is the case or not.

My advice would be to stick to the basics, don't pay anybody who makes any kind of promise about result ranking and avoid like the plague any scheme that is "unbeatable" and promises unrivalled PR within only a month or two.

Tuesday, 10 September 2013

New Version of Strictly AutoTags - Version 2.8.6

Strictly Auto Tags 2.8.6 Has Been Released!

Due to the severe lack of donations plus too many broken promises of "I'll pay if you just fix or add this" I am stopping to support the Strictly Auto Tags plugin.

The last free version, 2.8.5 is up on the WordPress repository: wordpress.org/plugins/strictly-autotags

It fixes a number of bugs and adds some new features such as:
  1. Updated storage array to store content between important content, bold, strong, headers and links etc. So they don't get tagged inside e.g put bolded words inside an existing h4 etc. 
  2. Changed storage array to run "RETURN" twice to handle nested code because of previous change.
  3. Fixed bug that wasn't showing the correct value for the minimum number of tags that a post must have before deeplinking to their tag page in admin. 
  4. Fixed bug in admin to allow noise words to have dots in them e.g for links like youtube.com 
  5. Added more default noise words to the list.
  6. Cleaned code that wasn't needed any-more due to changes with the way I handle href/src/title/alt attributes to prevent nested tagging.
  7. Removed unnecessary regular expressions which are not needed now. 
Version 2.8.5 of Strictly AutoTags www.strictly-software.com/plugins/strictly-auto-tags

Is going to be a "donate £40+" and get a copy version.

I am going to be sexing this plugin up into more of an SEO, text spinning, content cleaning, auto-blogging tool full of sex and violence in the future and I am running it on my own sites at the moment to see how well it does.

New features in 2.8.6 include.

Set a minimum length of characters for a tag to be used.

Set equivalent words to be used as tags. I have devised a "mark up" code for doing this which will allow you to add as many tag equivalents as you want. For example this is a cut down current version from one of my sites to show you an example using Edward Snowden (very topical at the moment!).

[NSA,Snowden,Prism,GCHQ]=[Police State]|[Snowden,Prism]=[Edward Snowden]|[Prism,XKeyscore,NSA Spying,NSA Internet surveillance]=[Internet Surveillance]|[TRAPWIRE,GCHQ,NSA Spying,Internet surveillance,XKeyscore,PRISM]=[Privacy]|[Snowden,Julian Assange,Bradley Manning,Sibel Edmonds,Thomas Drake]=[Whistleblower]

As you can see from that example you can use the same words multiple times and give them equivalent tags to use. So if the word Snowden appears a lot I will also tag the word "Police State", "Edward Snowden", "Whistleblower" as well as Snowden.

This feature is designed so that you can use related words as tags that maybe more relevant to peoples searches. 

I also have added a feature to convert textual links that may appear from importing or scraping into real links for example www.strictly-software.com will become a real link to that domain e.g http://www.strictly-software.com.

I have added the new attributes data and their derivatives e.g data-description or data-image-description , basically anything that has data- at the front of it, into my list of attributes to store and then replace after auto-tagging to prevent nested tags being added inside them.

I will be extending this plugin lots in the future but only people prepared to pay for it will be able to get the goodies. I am so fed up of open-source coding there is little point in me carrying on working my ass off for free for other peoples benefit any more.

If you want a copy email me and then I will respond.

You can then donate me the money and I will send you a unique copy.

Any re-distribution of the code will mean hacks, DDOS, viruses from hell and Trojans coming out your ass for years!





Testing Server Load Before Running Plugin Code On Wordpress

Testing Server Load Before Running Plugin Code On Wordpress

UPDATED - 10th Sep 2013

I have updated this function to handle issues on Windows machines in which the COM object might not be created due to security issues.

If you have a underpowered Linux server and run the bag of shite that is the WordPress CMS system on it then you will have spent ages trying to squeeze every bit of power and performance out of your machine.

You've probably already installed caching plugins at every level from WordPress to the Server and maybe even beyond....into the cloud....all stuff normal websites shouldn't have to do but it seems WordPress / Apache / PHP programmers love doing.

A fast optimised database, queries that return data in sets (not record by record) and some static pages for content that doesn't change constantly should be all you need but it seems that this is not the case in the world of WordPress!

Therefore if you have your own server or virtual server and the right permissions you might want to consider implementing some code in important plugins that prevents the job you intend running causing more performance problems if the server is already over loaded.

You can do this by testing for the current server load, setting a threshold limit and then only running the code you want if the server load is below that limit.

Of course security is key so lock down permissions to your apps and only let admin or the system itself run the code - never a user and never by a querystring that could be hacked!

The code is pretty simple.

It does a split for Windows and non Windows machines and then it checks for a way to test the server load in each branch.

For Windows it has two methods, one for old PHP code and one for PHP 5+.

In the Linux branch it tests for access to the /proc/loadavg file which contains the current load average on LINUX machines.

If it's not there it tries to access the shell_exec function (which may or may not be locked down due to permissions - up to you whether you allow access or not) and if it can run shell commands it calls the "uptime" function to get the current server load from that.

You can then call this function in whatever plugin or function you want and make sure your server isn't already overloaded before running a big job.

I already use it in all my own plugins, the Strictly Google Sitemap and my own version of the WP-O-Matic plugin.


/**
 * Checks the current server load
 *
 * @param boolean $win
 * @return string 
 *
 */
function GetServerLoad(){

 $os = strtolower(PHP_OS); 
 
 // handle non windows machines
 if(substr(PHP_OS, 0, 3) !== 'WIN'){
  if(file_exists("/proc/loadavg")) {    
   $load = file_get_contents("/proc/loadavg"); 
   $load = explode(' ', $load);     
   return $load[0]; 
  }elseif(function_exists("shell_exec")) {     
   $load = @shell_exec("uptime");
   $load = explode(' ', $load);        
   return $load[count($load)-3]; 
  }else { 
   return false; 
  } 
 // handle windows servers
 }else{ 
  if(class_exists("COM")) {     
   $wmi  = new COM("WinMgmts:\\\\."); 
   if(is_object($wmi)){
    $cpus  = $wmi->InstancesOf("Win32_Processor"); 
    $cpuload = 0; 
    $i   = 0;   
    // Old PHP
    if(version_compare('4.50.0', PHP_VERSION) == 1) { 
     // PHP 4      
     while ($cpu = $cpus->Next()) { 
      $cpuload += $cpu->LoadPercentage; 
      $i++; 
     } 
    } else { 
     // PHP 5      
     foreach($cpus as $cpu) { 
      $cpuload += $cpu->LoadPercentage; 
      $i++; 
     } 
    } 
    $cpuload = round($cpuload / $i, 2); 
    return "$cpuload%"; 
   }
  } 
  return false;     
 } 
}

A simple server load testing function that should work across Windows and Linux machines for load testing.

Saturday, 31 August 2013

Displaying Apache Server Status HTML

Display Your Apache Server Status As HTML Webpage

Diagnosing and fixing Apache issues can be a fucking nightmare (excuse the french). However one thing you might want to do is allow yourself access to view key Apache stats on a webpage rather than have to SSH in and use the console.

If you read some articles on the web to set this up it's as simple as adding the following lines to your main apache2.conf or httpd.conf file.

Depending on your server you will need to go into /etc/apache2/apache2.conf or etc/httpd/conf/httpd.conf and then adding the following lines.

# Allow server status reports, with the URL of http://dv-example.com/server-status
# Change the ".dv-example.com" to match your domain to enable.
ExtendedStatus on

   SetHandler server-status
   Order Deny,Allow
   Deny from all
   Allow from mywebsite.com

Or if you wanted to only allow your own IP address to access the page you could do something like this.

# Allow server status reports, with the IP addresss you want to allow access
ExtendedStatus on

   SetHandler server-status
   Order Deny,Allow
   Deny from all
   Allow from 86.42.219.12

However when I tried either of these approaches I was just met with an error message:

Forbidden

You don't have permission to access /server-status on this server.




The Fix
At the bottom of my main apache2.conf file I noticed the lines
# Include generic snippets of statements
Include /etc/apache2/conf.d/

Therefore I went into that folder and was met with 3 other files:

apache2-doc
charset
security

The top of the security file had these lines.


# Disable access to the entire file system except for the directories that
# are explicitly allowed later.
#
# This currently breaks the configurations that come with some web application
# Debian packages. It will be made the default for the release after lenny.
#
#
# AllowOverride None
# Order Deny,Allow
# Deny from all
#

So it seems that the main file was loading in these other files and the security file was blocking access to all other directories. By adding my rule to the top conf file the security file just over-ruled it again which caused the 403 error. Therefore I added the rules to the bottom of the security file and hey presto it worked.

Now I can access one of my domains to find my apache info. However I found that due to most of my sites on this virtual server using WordPress their ISAPI rules prevent a simple mysite.com/server-status from working.

WordPress is obviously trying to resolve the URL to a page, post or category list.

Therefore if you have got this problem you might need to use a domain not full of rules or create a rule that bypasses the WordPress guff.

Once you get it working you will get Apache info such as the following.
Current Time: Saturday, 31-Aug-2013 08:28:06 BST
Restart Time: Saturday, 31-Aug-2013 08:15:13 BST
Parent Server Generation: 0
Server uptime: 12 minutes 53 seconds
Total accesses: 622 - Total Traffic: 4.2 MB
CPU Usage: u26.65 s3.24 cu0 cs0 - 3.87% CPU load
.805 requests/sec - 5.5 kB/second - 6.9 kB/request
3 requests currently being processed, 4 idle workers

Plus lots of info about current HTTP requests, CPU usage and session info.

It is well worth doing if you need info about your server and you are away from your SSH console.

Thursday, 22 August 2013

Handle jQuery requests when you want to reference code that hasn't loaded yet

Handle jQuery requests when you want to reference code that hasn't loaded yet

As you should be aware it is best practise to load your JavaScripts at the bottom of your HTML for performance and to stop blocking or slow load times.

However sometimes you may want to reference an object that is not yet loaded higher up in the page.

If you are using a CMS or code that you cannot change then you may not be able to add your event handlers below any scripts that maybe needed to use them. This can cause errors such as:

Uncaught ReferenceError: $ is not defined 
Uncaught ReferenceError: jQuery is not defined

If you cannot move your code below where the script is loaded then you can make use of a little PageLoader function that you can pass any functions to and which will hold them until jQuery (or any other object) is loaded before running the functions.

A simple implementation of this would involve a setTimeout call that constantly polls the check function until your script has loaded.

For example:


PageLoader = { 
 
 // holds the callback function to run once jQuery has loaded if you are loading jQuery in the footer and your code is above
 jQueryOnLoad : function(){}, 

 // call this function with your onload function as the parameter
 CheckJQuery : function(func){
  var f = false;

  // has jQuery loaded yet?
  if(window.jQuery){
   f=true;
  }
  // if not we store the function if first time in loop otherwise and set a timeout
  if(!f){
   // if we have a function store it until jQuery has loaded
   if(typeof(func)=="function"){    
    PageLoader.jQueryOnLoad = func;
   }
   // keep looping until jQuery is in the DOM
   setTimeout(PageLoader.CheckJQuery,200);
  }else{
   // jQuery has loaded so call the function
   PageLoader.jQueryOnLoad.call();    
  }
 }
}


As you can see the object just holds the function passed to it in memory until jQuery has been loaded in the DOM. This will be apparent because window.jQuery will be true.

If the object isn't in the DOM yet then it just uses a setTimeout call to poll the function until it has loaded.

You could increase the length of time between the polls or even have a maximum limit so that it doesn't poll for ever and instead after 10 loops returns an error to the console. However this is just a simple example.

You would call the function by passing your jQuery referencing function to the CheckJQuery function like so.


<script>
PageLoader.CheckJQuery(function(){
 $("#myelement").bind('click', function(e) { 
  // some code
  alert("hello");
 });
});
</script>


It's just a simple way to overcome a common problem where you cannot move your code about due to system limitations but require access to an object that will be loaded later on.