Wednesday, 31 August 2011

Would you switch back to IE9 now that they have finally made a good browser?

Now that IE9 is standards compliant and actually good will you switch back to using it?

A few months ago I wrote an article about why I hated debugging in IE8 which revolved around their single threaded event model (window.event) and the delay it took before debug messages were outputted to the console.

A member of the Microsoft team commented that he had run the test case I had created in IE9 and that it had run perfectly. No high CPU, no frozen console, no high memory usage or other performance problems at all.

As you cannot install IE9 on WinXP, which is what I use on my home laptop (due to Vista being shite), I haven't had the pleasure of using Internet Explorer 9 a lot until I installed Windows 7 on my work PC.

I have to say that Windows 7 is actually a great operating system and I especially love the way it has incorporated many of the features of clean up and performance tuning tools like Tune Up Utilities and CCleaner into the main OS.

I also have to say that Internet Explorer 9 is the first IE browser I actually like.

Not only is it blindingly fast they have finally listened to their customers who have been complaining for years and made their JavaScript engine standards compliant.

Obviously this makes the Browser / Document mode spaghetti even harder to detect and I haven't been able to find a way as of yet to detect IE 9 browser mode running as IE 8 or 7 on 32 bit machines but that is not a major issue at all.

What I am wondering though is that now that IE9 is actually a good browser, with a good inbuilt development console and element inspection tools, how many developers will actually return to using it either as their browser of choice for surfing the web or for their primary development.

My browser development survey which I held before IE9 was released showed that developers would rather use Chrome or Firefox for surfing and would also always choose the development tools that those browsers bring than use IE 7 or 8.

I know that I changed from using IE to Firefox as my browser of choice some eons back and I changed from Firefox to Chrome for both surfing (speed is key) and developing (no need for Firebug or any other plugin) the other year when Firefox started to suffer major performance problems.

These performance problems are always either to do with having far too many plugins installed. Setting the browser up to check for new versions and any installed plugins on start up which cause long load times, Firebug issues and errors in the Chrome source. This is on top of all the constant problems that Flash seems to bring to any browser.

I use Chrome for surfing due to it's speed but if I leave a few tabs open all night that contain pages running Flash videos then by morning my CPU has flat-lined and memory has leaked like the Titanic.

This is more a problem with Flash than Chrome and I try to keep this browser for surfing as clean and as fast as possible by not installing plugins. Then if I need to hack around or do some hardcore web development I will use Firefox and all the great plugins when I need to.

I don't think I could really get hacking with Chrome alone as when I am really getting stuck into a site I need my key plugins e.g the web developer toolbar, hackbar, header modifier, Request and Response inspectors e.g HTTP Fox, Firebug, ColorZilla, YSlow and all the Proxy plugins that are available.

However for basic development, piping out messages to the console, inspecting the DOM and checking loaded elements and their responses then Chrome has everything Firebug has without all the JavaScript errors.

In fact it's surprising how many developers don't even realise how much is possible with Chrome's inbuilt developer tools, speed tests, header inspectors, element inspectors and DOM modifiers and other great debugging tools that used to be the preserve of Firebug alone.

Which leads me back to IE9.

Yes it's fast and it's developer tools are okay with options to clear cookies, disable JavaScript, inspect the DOM and view the styles but it's no Chrome or Firebug yet;

Therefore what I want to know is how many people (and by that I mean developers) will switch back to using Internet Explorer 9 for pure web surfing?

For sure it's fast, supports the latest CSS standards and all the great HTML 5 features that we have all seen with floating fish tanks and the like. But is this enough for another browser switch?

I have all the major browsers installed at work on Win7 including IE9, Firefox, Chrome, Opera, Safari, SeaMonkey and a number of others I cannot even remember the names of they are that obscure.

Before upgrading I even had Lynx and Netscape Navigator 4 installed as I used to be so anal that any code such as floating DIV's had to work in the earliest browsers possible. However I only use these browsers for testing and I stick to Chrome (for surfing) and Firefox (for hacking) which leaves little room for IE9 at the moment.

I know it's a good browser. It's standards compliant and it no longer crashes when I try to output anything to the console so why shouldn't I try it again.

Maybe one of my readers can help me decide on the browser of choice for surfing and development and how IE9 fits into that toolset.

Maybe I am missing some really cool features that I would use all the time but to be honest I am not into throwing sheep at randoms or constantly voting sites up and down and writing comments on them. Therefore for me to switch back from Chrome to another browser such as IE9 there has to be a really good reason for me to do so.

So does anyone have any reason why I should re-consider using IE9 at work for my web surfing or development?

Wednesday, 3 August 2011

HTML Super Trim Functions

Remove non breaking spaces and other white space either side of strings

Most languages have a Trim function that removes white space from either side of a string.

However a lot of the time these Trim functions will only remove standard white space e.g if you hit the space bar a couple of times and not other forms of white space such as tabs, new lines or HTML space characters such as non breaking spaces whether they are HTML entity encoded: or Numerically encoded e.g

Therefore sometimes you may need a "Super Trim" function that will handle the removal of all types of space characters including HTML entities.

The following have been written in PHP but can easily be converted into C# or VB. The main part to take away is the regular expression used within each function which replaces a string containing one or more space characters whether they be control characters or HTML entities from either side of the string.

// wrapper function to do trim both sides
function HTMLTrim($text){

// call both functions at once
return HTMLLeftTrim(HTMLRightTrim($text));
}

// removes spaces and at the beginning of strings
function HTMLLeftTrim($text){

// remove space to the left of the text
return preg_replace("@^( | |\s)+(\S+)@","$2",$text);

}

// removes spaces and at the beginning of strings
function HTMLRightTrim($text){

// remove space to the right of the text
return preg_replace("@(\S+)( | |\s)+$@","$1",$text);

}


You can test this out in a simple PHP page with the following code:


$str = "     hello there        ";

echo "before trim its '" . $str . "'";

echo "<br><br>now its '" . HTMLTrim($str) . "'";


Which returns the following output:

before trim its '     hello there      '

now its 'hello there'

I find it very useful when I am scraping content from the web and need to handle the removal of a mixture of standard spaces and HTML spaces.

Wednesday, 27 July 2011

My JavaScript HTML Encoder Function

Why did I write my JavaScript HTML Encoder object

One of my most popular scripts that continues to get downloaded a lot as well as spark many an email from companies and users is my HTML Encoder script that I wrote many a year back.

You can see the script in action here: www.strictly-software.com/htmlencoder

I was emailed another function that claimed to do the same work earlier and I just thought it would be interesting to other readers to know why I created the script in the first place and why many of the examples out on the net that claim to do the same job are just not up to scratch.

On many sites nowadays people are obtaining content from various sources through XML and RSS feeds and they often use JavasScript and AJAX to do part of the job. I know that when I was creating my football site www.hattrickheaven.com which was basically an exercise in using the Google AJAX API I came across the following problems:

  1. Content is obtained from a multitude of sources and there is no common standard that can be relied on 100%.
  2. Content is often mixed together through feed blenders like Yahoo Pipes or Blastcasta.
  3. Content is often loaded in on the fly using AJAX, a Scraper Proxy to do cross domain content jacks and other formatting duties such as on the fly translations etc.
  4. There is no built in function for HTML encoding in the JavaScript language.

Now I looked at many an HTML Encode solution before writing my own but the main problem I needed to overcome was that content could already be HTML encoded or partially HTML encoded and functions I came across did nothing to handle this problem.

For example if you had this bit of text you will notice that it has an encoded ampersand in the middle but the quotes are not encoded.

"Rob says hello & goodbye"

If you ran this through many of the HTML Encode functions out there including many server side ones it will double encode the & in the & so that you end up with this.

&quot;Rob says hello &amp;amp; goodbye&quot;

Now you may want this to happen but I doubt it.

Not if you want to run all your content through the same function without worrying about double encoding issues like this on your website.

This is why I wrote my HTML Encoder object as it puts into practise the technique I used whilst writing ASP classic sites that supported the UTF-8 character set.

With multi-lingual sites that display Arabic or Japanese character sets you don't want to be using the inbuilt Server.HTMLEncode function on all your textual input as it will triple the size of everything you store by converting every non ASCII character into a &#XXXX; encoded string.

When your database is set up to correctly store your text as Unicode (nvarchar etc) and you are outputting it with the correct code-base then you need something a bit more clever to do the job as you still need to encode characters that can cause you damage such as the naughty 4 " ' < > as well as the ampersand & to make your page validate (URL's, Querystrings etc).

This is obviously for security reasons as you don't want people to be able to malform your input and break your layout or insert XSS hack vectors into your system.

However running the standard HTML Encode function will encode everything as well as cause double encoding issues. Therefore the HTML Encoding needs to be done in stages and the ampersand has to be handled correctly as it makes up part of the HTML encoded &#XXXX; format.

So whilst it comes as quite a surprise to see how popular this script is (#1 on Google for "HTML Encode with Javascript") it is not really surprising when one thinks about the problem this script solves.

Whilst I don't claim to be any kind of coding genius and I am sure the code can be improved it does do what is says on the tin and if you are thinking of using another JavaScript function to HTML Encode your content then compare it first with my tool to see if it handles the double encoding issue as if it doesn't then you might find yourself having problems down the road.

Tuesday, 26 July 2011

Wordpress posts_per_page Query Option Not Working

Fixing the posts_per_page query parameter bug in Carrington Themes

This is just a quick but annoying bug fix for anyone using the Carrington Theme for Wordpress.

If you are creating your own custom templates or pages and want to create your own custom query for the loop then you will probably want to specify how many articles to show.

This is determined by the posts_per_page query filter which determines how many articles are displayed during the loop output e.g

query_posts('cat=234&posts_per_page=5); 

However when I was trying to create a custom template earlier this value was being ignored and I couldn't work out why.

I eventually stumbled across a thread on the Wordpress forums that gave me the solution and it relates to a "feature" in the Carrington theme. Apparently it creates it's own function which ignores any value you may give the posts_per_page query option.

The solution is to disable this "feature" by removing the filter that sets it in motion. Once this is done your loop queries will take into account any value you give the posts_per_page query option.

You can disable this feature on the page you are building by adding the following code before creating your query loop.

remove_filter('pre_get_posts', 'cfct_posts_per_archive_page');


A full example of a custom template page based on the Carrington theme that uses this feature can be seen below.


<?php
/*
Template Name: SomethingTemplate
*/

if (__FILE__ == $_SERVER['SCRIPT_FILENAME']) { die(); }
if (CFCT_DEBUG) { cfct_banner(__FILE__); }

get_header();

?>

<div id="content">

<?php


// get category id by name - if you know it then skip this and add it directly to the query
$id = get_cat_id('Some Category');

// remove the carrington blog filter that overwrites the posts_per_page query filter parameter
remove_filter('pre_get_posts', 'cfct_posts_per_archive_page');

// create a query to filter by the desired category and show the number of posts as specified in the global settings and handle paging
query_posts('cat='.$id.'&posts_per_page='.get_option('posts_per_page') . '&paged='.$paged);


if (have_posts()) {
while (have_posts()) {
the_post();?>
<h1><a href="<?php the_permalink(); ?>" title="This is my blog article: <?php htmlspecialchars(the_title()) ?>"><?php the_title(); ?></a></h1>

<?php the_content(); ?>

<?php
}
}

?>

<div class="pagination">
<span class="previous"><?php previous_posts_link('« Previous') ?></span>
<span class="next"><?php next_posts_link('Next »') ?></span>
</div>

</div>

Wednesday, 13 July 2011

PHP for obtaining the follower count of a Twitter Account

Get Twitter Follower Count using Regular Expressions

I came across this bit of PHP code the other day which is aimed at getting the follower count of a Twitter user.

It seems like overkill to me and is a mixture of regular expressions, string parsing, callback functions and a lot of head scratching.

The user obviously knows that the follower count HAS to reside within an element within the DOM with the id of follower_count so why not just use one single regular expression to target that element and return it's guts instead of all the DOM loading, callbacks and string parsing?

I might be missing something that someone could tell me but this seemed like a long way to go about a simple scrape job.


// Get the number of twitter followers

function string_getInsertedString($long_string,$short_string,$is_html=false){
if($short_string>=strlen($long_string))return false;
$insertion_length=strlen($long_string)-strlen($short_string);
for($i=0;$i<strlen($short_string);++$i){
if($long_string[$i]!=$short_string[$i])break;
}
$inserted_string=substr($long_string,$i,$insertion_length);
if($is_html && $inserted_string[$insertion_length-1]=='<'){
$inserted_string='<'.substr($inserted_string,0,$insertion_length-1);
}
return $inserted_string;
}

function DOMElement_getOuterHTML($document,$element){
$html=$document->saveHTML();
$element->parentNode->removeChild($element);
$html2=$document->saveHTML();
return string_getInsertedString($html,$html2,true);
}

function getFollowers($username){
$x = file_get_contents("http://twitter.com/".$username);
$doc = new DomDocument;
@$doc->loadHTML($x);
$ele = $doc->getElementById('follower_count');
$innerHTML=preg_replace('/^<[^>]*>(.*)<[^>]*>$/',"\\1",DOMElement_getOuterHTML($doc,$ele));
return $innerHTML;
}


// To display it

<?php echo getFollowers("username"); ?>



Here is the much shorter version I wrote. It still works just as well returning the follower count of the Twitter Account username passed into it.

function getFollowers($username){
$url = "http://twitter.com/".$username;
$count=0;

$x = file_get_contents($url);

preg_match("@<([a-z][^ ]*) id=\"follower_count\"[^>]+?>([0-9,]+)\s*</\\1>@i",$x,$match);

if($match){
$count = $match[2];
}

return $count;
}

// lets check how poorly my twitter account is followed!
echo "StrictlyTweets: " . getFollowers("StrictlyTweets") . "<br><br>";



As you can see from the regular expression I am matching HTML tags (they all start with < and then a letter) and storing that tag to be used in the backtrack reference later on so that if the HTML changes from a SPAN to a DIV as long as it has the id="follower_count" with the element there will be a match.

I could have loaded up the DOM, targeted the ID and then done some regex but why bother when you can go straight for the juggular!

Tuesday, 12 July 2011

Firefox 5.0 Firebug 1.8 Memory Leak Problem

Firefox 5.00 with Firebug 1.8.0b5 on Windows 7 64 bit causes Memory Leak

I came across a problem with Firefox 5.0 and Firebug 1.8.0b5 today that basically leaked memory like an open oil well and hung my PC every time I tried to open up a basic popup window from a page using Javascript.

My PC is running the following:

Windows 7 - Dual Core - 64 Bit
Firefox.exe *32 - version 5.0
Firebug 1.8.0b5

This problem has only started occurring since my latest upgrade and I narrowed it down to Firebug by disabling each plugin one by one.

I can replicate the problem on one particular page quite easily which has a link with a simple javascript: window.open(url) function that opens up a little static html page.

When I now click the link the popup opens full size rather than the dimensions set 400px * 400px and there is nothing on the page - just a blank screen. Viewing the source shows nothing at all.

This page used to work fine before the update.

It still works on all other browsers Chrome, Safari, IE (6 to 9) and it still works in Firefox 5.0 on 32 bit machines so I am not sure whether it's a 64 bit problem or not.

Viewing my Task Manager window I can open Firefox and I can get my Firefox to use beyond 1GB of memory and 50% CPU (the whole of one of my two processors) with only one single window and tab open all within 5 minutes.

Earlier today it was using over 3GB of memory!

Another clue is the fact that the error console is permanently full of these error messages:

attempt to run compile-and-go on a cleared scope
resource://firebug_rjs/console/errors.js
line 156

Hitting clear just fills up the whole of Firebug's console log instantly with the same message.

The only way I have found to solve this problem is to disable Firebug and not use the "Restart Firefox" link as this seems to just ramp the memory up without shutting the process down first.

I usually have to kill the process through Task Manager as clicking the close button doesn't do anything the majority of the time.

If anyone else has similar problems please let me know.


Saturday, 9 July 2011

URLEncode Problem with .NET 4.0

Visual Studio 2010 Problem with "The name HttpUtility does not exist in the current context".

I usually use Visual Studio 2010 at work and the Express edition at home. Tonight I was knocking up quick C# windows form application to crawl some specific URL's and I required the ability to URL Encode my URI Components e.g

string url = "http//somesite.com/search?qry=" +  System.Web.HttpUtility.UrlEncode(val);

However even though I had included:

using System.Web;

At the top of my class it was saying that HttpUtility could not be found in System.Web.

This was driving me crazy as I had another project open at the same time which was using the exact same code and it worked.

The advice I found on the web said to ensure that a reference to System.Web was added with the "Project" > "Add References" option.

However when I was searching for this DLL all I could find were System.WebServices and System.ApplicationServices which were obviously no good.

However after a lot of head scratching I went back to the main Project Properties panel and under Target Framework I had ".NET Framework 4 Client Profile" Selected.

I changed this first to .NET Framework 3.5 and then looked again for the reference and it worked!

I added it and then changed it back again to .NET Framework 4. On re-opening the project  all my problems had been solved.

Don't ask me why System.Web doesn't appear as a reference under the Client versions but it doesn't and you need to add a proper DLL reference to get to use this sort of functionality if you are doing non web based apps that require web based functionality like URL or HTML Encode etc.

I thought I would just write a few notes about this in-case anyone else experiences the same problem.