Microsoft's new version of Internet Explorer, IE 8 comes with two configuration settings that allow the user to control the rendering mode and the browser mode so that in fact IE 8 comes with 6 different settings that can drastically alter the behaviour of the browser. See this article about the different settings for an overview.
If you want to be able to determine client side what configuration mix is currently enabled for the browser you can use the following function I have written which will output a little object with the relevant details. Either use it by itself or incorporate it into a fuller browser detection object such as my browser object.
If you are running IE 8 you will see below the output from the function:
download the script here: IE 8 compatibility mode detection script
How to detect which settings the user has enabled in IE 8?
To determine the browser mode you can test the user-agent which will mention Trident/4.0 for IE 8, and IE 8 in compatibility mode. IE 8 running as IE 7 will not mention this so combining this with a check for document.documentMode which is only available in IE 8 will enable you tell if the user has IE 7 or is running IE 8 as IE 7.
To determine the document mode you can test the new property which is available in IE 8 document.documentMode which will return the rendering mode its currently set to e.g 8, 7 or 5 (for quirks mode). Combining this with a check for document.compatMode for previous IE versions lets us determine the rendering engine settings.