The Strictly iPhone Console - Increasing real debugging without simulators



Yesterday I blogged about debugging on iPhones and how useful the Debugger console was which showed up JavaScript errors and console messages without the need for an agent switcher. However I complained about the lack of ability to view the generated source code on the page when viewing on an iPhone.

However I remembered sometime back I blogged about some bookmarklets I was using that enabled me to view the generated source on older browsers like IE 6 and tonight I knocked together a little script that enables you to have basic "onscreen" debugging functionality when using an iPhone.

The code is some basic JavaScript that adds a DIV area to the bottom of the current page with two links at the top that allow you to view the generated and raw source code for the page. These open up in new windows and  if you have problems with them opening you might need to enable the popup window functionality in your iPhone Safari settings first.

Underneath is a basic console which due to the lack of scrollbar functionality on DIVs with overflow:auto or overflow:scroll I have created with a readonly textarea. If you need to scroll down the console you should use the two finger drag option to move the content within the textarea up or down.

I have also overwritten the window.console object so that the console.log function pipes out messages to this console if it's being used.

To Install the Strictly iPhone Console

1. On your iPhone visit this page and copy the JavaScript code from the area below into your clipboard. or vis

2 .Bookmark any page on your phone and then go into your bookmarks and edit it. Change the name to "Strictly Console" before pasting in the copied source code as the URL location for the bookmark.

3. Test that the code is working by going to a webpage on your iPhone and once the page has loaded open your bookmarks and select the "Strictly Console" bookmark. The console should appear at the bottom of the page. At the top of the console will be two links in a grey background "View Source" and "View Generated Source". Underneath will be the console area.

Clicking on either of those links will open up a new page with either the original source code or the generated source code from the current page. If nothing happens when you click the link check your Safari settings so that you allow Pop Ups. When you click the link it will ask you whether you want to open the pop up or not. Choosing Yes will show you the source in a new window.

4. To pipe debug out to the Strictly iPhone console window just use the standard console.log('hello'); function to do so.

As with all code that relies on DOM element make sure you check for the the existence of the console before trying to access it e.g a simple debug function could look like this:

function ShowDebug(m){
 if(typeof(window.console)!="undefined")){
  console.log(m);
 }
}

If you have problems copying and pasting from here (due to the crumy formatting of HTML in blogger) then you can download the compressed script from this location: iphoneconsolebookmark.js

javascript:(function()%7B
function%20htmlEscape(s)%7Bs=s.replace(/&/g,'&');s=s.replace(/%3E/g,'>');s=s.replace(/%3C/g,'<');return%20s%7D
var%20_d=document,jq=_d.createElement('script');jq.type='text/javascript';jq.async=true;jq.src='http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js';var%20s=_d.getElementsByTagName('script')[0];s.parentNode.insertBefore(jq,s);
console=%7B
log:function(m)%7var%20l=_d.getElementById("logwindow"),txt=l.value,nt=((txt==""||/^\s+$/.test(txt))?m:txt+m)+"\n";l.value=nt;%7D,
GenSource:function()%7Bconsole.ViewSource(htmlEscape('%3Chtml%3E\n'+_d.documentElement.innerHTML+'\n%3C/html%3E'))%7D,
RawSource:function()%7B$.get(location.href,function(data)%7Bconsole.ViewSource(htmlEscape(data))%7D)%7D,
ViewSource:function(s)%7Bx=window.open();x.document.write('%3Cpre%3E'+s+'\n%3C/pre%3E');x.document.close()%7D%7D;
var%20log=_d.createElement("div");log.setAttribute("id","logger");log.style.visibility="visible";log.style.display="block";log.style.zIndex="2147483647";log.style.position="relative";log.style.backgroundColor="#fff";log.style.border="1px%20solid";log.style.width="98%";log.style.left="0";log.style.bottom="0";log.style.height="3300px";log.style.padding="5px";log.style.textAlign="left";_d.getElementsByTagName("body")[0].appendChild(log);var%20link=_d.createElement("div");link.style.width="100%";link.style.background="lightgray";link.style.color="navy";link.style.fontWeight="bold";link.innerHTML="%3Ca%20onmouseout='this.style.color=\"navy\";'%20onmouseover='this.style.color=\"blue\";'%20style='text-decoration:none;'%20href='#'%20onclick='console.RawSource();return%20false;'%3EView%20Source%3C/a%3E%20|%20%3Ca%20onmouseout='this.style.color=\"navy\";'%20onmouseover='this.style.color=\"blue\";'%20style='text-decoration:none;'%20href='#'%20onclick='console.GenSource();return%20false;'%3EView%20Generated%20Source%3C/a%3E";log.appendChild(link);var txt=_d.createElement("textarea");txt.setAttribute("id","logwindow");txt.setAttribute("readonly","readonly");txt.style.width="100%";txt.style.height="300px";log.appendChild(txt);
%7D)();

Test the Strictly Debug Console now

You can try the Strictly iPhone debug console out on this page by clicking the following button which will add a new debug message to the console on each click. The console should already have 15 messages inside it which were added when the page loaded. Remember the console will be at the very bottom of the screen so scroll right down to see it.



Obviously this is a very basic iPhone Console and nothing like Firebug or Chromes inbuilt console but it could be easily expanded with a little work and by loading in the jQuery iphone library you could easily create a popup DOM inspector that was initialised by a long tap down event to show the current elements styling and positioning. Let me know what you think and if you amend it to add more features let me know so I can update the code here.

To read why a proper debugging console ON the device is required rather than a user-agent switcher then read this article I wrote about debugging on iPhones.

Tweet

Identifying WIN only markets on the BETFAIR API

Updated 02-JUN-2012 

As it's Derby day again I ran into the same problem as last year with this article except that the wrong market identified as WIN ONLY was the FAV SP market. One in which you bet on the favourites starting price. Therefore I have updated the BOT code for identifying a market from the Betfair API from HorseName, Racedatetime and Course alone.

If you don't know I developed the www.fromthestables.com website which allows members to access UK horse trainer information everyday about their runners.

As a side line I have also developed my own AutoBOT which uses the BETFAIR Free API to place bets automatically using my own ranking system which I have developed. You can follow my tips on Twitter at @HorseRaceInfo.

One of the problems I have come across during the development of my AutoBOT is that if you have the name of the Horse, Course and time of the race and want to find the Market ID that Betfair uses to identify each race there is a costly mistake that can occur due to all the various markets that are available.

I really got hooked on racing (not betting but actually watching horse racing) when I had a bet on Workforce in the 2010 Derby.

It came from the back of the field to storm past everyone else and won the Derby in record course time and in astonishing style.

Watching him apply the same tactics in the Prix de l'Arc de Triomphe to become the champion of Europe that same year installed the racing bug and then watching Frankel win the 2000 guineas this year in such amazing style has ensured that something I used to have no interest in watching whatsoever has become a TV channel turner.

Therefore when Frankel won the St Jame's Palace Stakes this year at Royal Ascot I was happy knowing that the AutoBOT I had written had placed a WIN bet on this horse early enough to get a decent price (for what was on offer for an almost 100% guaranteed win).

However when I found out that I had actually lost this bet that my BOT had placed I spent more than a few minutes scratching my head and cursing the PC I was sat in front of. However I found out that the actual market my application had put the bet on was a special WIN market in which the winner had to win by at least 4 clear lengths. Because Frankel had won by less than a length I had lost the bet. I wanted to know why.

I was annoyed.

I was quite pissed off actually and when I looked into it I found that to place a WIN only bet on the main WIN market in Betfair is quite a pain in the arse to achieve if you don't know the Market ID upfront as there is nothing in the compressed data that is given to you to identify that the market is the main WIN market and not some special market such as the one in which I lost that bet in.

Instead all you can do is run through a series of negative tests to ensure that the market is not a PLACE market, a Reverse Forecast or a Horse A versus Horse B market.

In fact since then I have found that there are so many possible markets it can be quite a nightmare to get the right one if you don't already have the Market ID.

For example today at 15:50 there was a race at Ascot, the Betfair Summer Double First Leg International Stakes that actually had alongside the usual markets a FIVE TO BE PLACED and TEN TO BE PLACED market. This was in a race with 23 runners!

The prices were obviously minimal and you would have had to of put down a tenner to win 70p on the favourite Hawkeythenoo but it meant that my original code to identify the main WIN market required updating as it was returning these new market ID's instead of the one that I wanted.

I have outputted the code for my Betfair API Unpack class below and this is just the part of my AutoBOT that returns a Market ID when provided with the compressed string of data that Betfair provides along with the Course name, the market type (WIN or PLACE) and the Race Date and Time.

You will see that I am using LINQ to filter out my data and I am using a custom function in my WHERE clause to return a match. It is this function that is the key as it has to check all the possible Betfair Market types to rule them out when looking for the main WIN market.

If you don't use C# then LINQ is one of the cool tools that makes it such a great language as it enables you to apply SQL like queries to any type of object that extends IEnumerable.

Obviously if you don't bet or don't use Betfair you might be wondering what the heck this has to interest you and you would be right apart from this bit of code being a nice example of how to use LINQ to return a custom list that can be iterated through like any array or list of objects.

Remember: Betfair may introduce even more markets in the future and if anyone knows of any markets I have missed then please let me know as I don't want to lose any more money by accident because of some weird market Betfair decides to trade on.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace BetfairUnpack
{
 // This is my object that holds data about the Betfair market
 public class MarketDataType
 {
     public int marketId;
     public string marketName;
     public string marketType;
     public string marketStatus;
     public DateTime eventDate;
     public string menuPath;
     public string eventHeirachy;
     public int betDelay;
     public int exchangeId;
     public string countryCode;
     public DateTime lastRefresh;
     public int noOfRunners;
     public int noOfWinners;
     public double totalAmountMatched;
     public bool bspMarket;
     public bool turningInPlay;
 }

 public class UnpackMarket
 {

     // Use my own class amd make a list object we can loop through like an array
     public List<MarketDataType> marketData;

     private string BaseDateVal = "1/1/1970";
     private string ColonCode = "&%^@"; // The substitute code for "\:"
     private int DaylightSavings = 3600000;

// This method unpacks a compressed string and returns the correct MarketID filtering by the Course, Date and Market type
     public UnpackMarket(string MarketString, string racecourse, DateTime racedatetime, string marketType)
     {

         string[] Mdata;

    // Betfair uses it's own format and we need to split on a colon
         Mdata = MarketString.Replace(@"\:", ColonCode).Split(':');

    // get our date and time
         DateTime BaseDate = Convert.ToDateTime(BaseDateVal);

         // if we are not currently in daylight savings then set that property to 0 so we get the correct time
    // I have had instances where the correct market is not returned due to Daylight savings time
         if (!DateTime.Now.IsDaylightSavingTime())
         {
             DaylightSavings = 0;
         }

    // Use LINQ on our IEnumerable object to query our list of markets filtering by our custom function MatchMarket
         IEnumerable<MarketDataType> queryMarkets =
             from m in Mdata
             where !String.IsNullOrEmpty(m)
             let field = m.Split('~')
             where (MatchMarket(field[5], BaseDate.AddMilliseconds(DaylightSavings + Convert.ToDouble(field[4])), field[1], racecourse, racedatetime, marketType))
             select new MarketDataType()
             {
                 marketId = Convert.ToInt32(field[0]),
                 marketName = field[1].Replace(ColonCode, ":"),
                 marketType = field[2],
                 marketStatus = field[3],
                 eventDate = BaseDate.AddMilliseconds(DaylightSavings + Convert.ToDouble(field[4])),
                 menuPath = field[5].Replace(ColonCode, ":"),
                 eventHeirachy = field[6],
                 betDelay = Convert.ToInt32(field[7]),
                 exchangeId = Convert.ToInt32(field[8]),
                 countryCode = field[9],
                 lastRefresh = BaseDate.AddMilliseconds(DaylightSavings + Convert.ToDouble(field[10])),
                 noOfRunners = Convert.ToInt32(field[11]),
                 noOfWinners = Convert.ToInt32(field[12]),
                 totalAmountMatched = Convert.ToDouble(field[13]),
                 bspMarket = (field[14] == "Y"),
                 turningInPlay = (field[15] == "Y")
             };

    // convert into a nice easy to iterate list
         marketData = queryMarkets.ToList();

     }

// return a Market if the values provided match
     private bool MatchMarket(string menuPath, DateTime eventDate, string marketName, string racecourse, DateTime racedatetime, string marketType)
     {
         bool success = false;

    // do some cleaning as Betfair's format isn't the prettiest!
         menuPath = menuPath.Replace(ColonCode, ":");
         marketName = marketName.Trim();

         // does the path contain the market abreviation - we keep a list of Courses and their Betfair abreviation code
         if (menuPath.Contains(racecourse))
         {
             // check the date is also in the string
             string day = racedatetime.Day.ToString();
             string month = racedatetime.ToString("MMM");

             // we don't want 15:00 matching 17:15:00 so add :00 to the end of our time
             string time = racedatetime.ToString("HH:mm:ss");

             if (menuPath.Contains(day) && menuPath.Contains(month) && eventDate.ToString().Contains(time))
             {
                 // if no bet type supplied returned all types
                 if (String.IsNullOrEmpty(marketType))
                 {
                     success = true;
                 }
                 else
                 {
                     if (marketType == "PLACE")
                     {
                         // place bet so look for the To Be Placed market
                         if (marketName.Contains("To Be Placed"))
                         {
                             return true;
                         }
                         else
                         {
                             return false;
                         }
                     }
                     // we can only identify the main WIN market by ruling out all other possibilities if Betfair adds new markets then this
  // can cost us some severe money!
                     else if (marketType == "WIN")                                                                                                                                                                                                                                                  
                     {
      // rule out all the various PLACE markets which seem to go up to ten horses!
                         if (marketName.Contains("To Be Placed") || marketName.Contains("Place Market") || marketName.Contains("TWO TBP") || marketName.Contains("THREE TBP") || marketName.Contains("FOUR TBP") || marketName.Contains("FIVE TBP") || marketName.Contains("SIX TBP") || marketName.Contains("SEVEN TBP") || marketName.Contains("EIGHT TBP") || marketName.Contains("NINE TBP") || marketName.Contains("TEN TBP"))
                         {
                             return false;
                         }
                         // ignore forecast & reverse forecast and horseA v horseB markets                            
                         else if (marketName.Contains("forecast") || marketName.Contains("reverse") || marketName.Contains(" v ") || marketName.Contains("without ") || marketName.Contains("winning stall") || marketName.Contains(" vs ") || marketName.Contains(" rfc ") || marketName.Contains(" fc ") || marketName.Contains("less than") || marketName.Contains("more than") || marketName.Contains("lengths") || marketName.Contains("winning dist") || marketName.Contains("top jockey") || marketName.Contains("dist") || marketName.Contains("finish") || marketName.Contains("isp %") || marketName.Contains("irish") || marketName.Contains("french") || marketName.Contains("welsh") || marketName.Contains("australian") || marketName.Contains("italian") || marketName.Contains("winbsp") || marketName.Contains("fav sp") || marketName.Contains("hcap") || marketName.Contains("the field"))
                         {
                             return false;
                         }
                         else
                         {
                             return true;
                         }
                     }
                     else
                     {
                         // I cannot match anything!
                         return false;
                     }

                 }
             }
         }

         return success;
     }
 }
}

Tweet

Solving the Demand 5, Flash Movie problem - and other Catch Up TV website issues

If you are from the UK then you may often like to catch up on missed TV programmes by watching the online TV catch up services such as BBC iPlayer, Channel Four's 4OD and Channel 5's TV Catch Up service called Demand 5.

The past few nights I have been trying to catch up on a number of programmes shown on Channel 5 as for some reason Five Star has stopped showing the latest episode of Burn Notice on a Sunday afternoon and I like to watch other shows (when they are available) like NCIS and The Mentalist etc so I try to use Channel 5's catch up website Demand 5.

I really wish they would buy the rights to show for longer so that you could watch a programme for more than just 7 days but then it's not called catchup TV for nothing and most TV programmes are made in the USA.

However when I use the Demand 5 website to watch the programme I experience a very annoying problem which effects every browser and computer I tried using including Chrome, FireFox, Safari and IE 8 on a Sony Vaio Windows XP 32 bit Dual Core and IE 9 on a Dell Windows 7 64 bit Quad core.

The Demand 5 catchup TV problem means that pre-show adverts play fine and then after a period of anything between a few seconds and a couple of minutes of the actual show playing the Demand 5 Catchup TV programme would either just freeze up and not start playing again at all. 

Or it would play for a while before stopping for a few seconds as if it was buffering more content before playing for another few seconds again - totally unwatchable.

I couldn't even move the skip bar backwards or forward and the pause / play button just didn't work at all.

A refresh would reload the page, show the adverts before the programme again and then the problem would re-occur. It was all very annoying..

After a number of seemingly ignored comments to Demand 5 complaining about them not fixing the problem and after reading on almost every show on the site that a large number of other people have had similar problems I tried solving the issue myself.

If you go to any show on Demand 5 and read the comments at the bottom e.g www.channel5.com/shows/ you will see comments along the following lines.

"Really poor transmission of episode 23. Can't get more than 20 seconds of coherent dialogue and then it freezes"

"Impossible to watch because of the poor service from Demand Five's website. Even worse than Karen's experience."

"Why is demand 5 not working? It hasn't worked for several weeks. You don't get this with iplayer!!!"

"doesnt seem to be able to play anything"

"wow the longest i got to watch before it froze was 50sec. i did give up 5min in though. I'm with sue off to iplayer which works."

And those are just a snapshot of the many comments on a couple of shows I wanted to watch today all complaining about the Demand 5 website video freezing or not playing at all. 

Demand 5 really need to fix this problem if they want to keep visitors coming to the site.

Due to my own comments and complaints not being answered I tried fixing the problem with Demand Five freezing myself - only because I really wanted to watch the latest episode of NCIS.

I wasn't going to step through all their custom JavaScript and I have no idea what they are doing server side so proper debugging is out of the question but what you can do in situations like this is turn everything else off and then back on again one by one to see if any section of the browser or website is causing the problem.

So without knowing whether they are running some client side code constantly that rewrites the DOM such as many sites try to use to beat all the plugins that are designed to hide adverts, images and flash I could try to see if turning off those parts of the site helped as they maybe running code to constantly re-insert adverts into the HTML to replace any removed by plugins like AdBlocker.

If they were doing something like this (and I don't know if the are) then it's possible that a race condition might occur between Demand 5's own advertising code and any any plugins or virus checkers (which have built in anti-banner options) that constantly scan and rebuild the DOM.Only proper debugging will prove if that is happening,

Debgging the Demand 5 website Performance Problem

So to get a fix for the the Demand 5 problem as soon as possible the first thing I did were all the normal things that developers try when met with similar issues. If you find similar problems on other sites or with plugins for websites you should always try this list first to rule them out as reasons for the problem.

• Disabling all cookies - the Web Developer Toolbar in Firefox is great for this and if you are not signing into the website there is no need for cookies to be enabled anyway.
• Disabling Java - Most websites don't even use Java Applets any-more so its not really needed until you find a site that actually makes use of it.
• Disabling JavaScript - Most websites that show TV content actually require JavaScript to be enabled for their site to work at all even though a basic Flash OBJECT or VIDEO element on a page outputted with server side code playing a movie doesn't require it. However companies like Demand 5 or iPlayer don't do this because it mean their content can easily be stolen or watched overseas by non UK citizens etc so JavaScript is actually required to load in the adverts and programming in chunks.
• Turn off all Flash tracking by disabling Flash cookies. You do this by right clicking on a flash movie, click the Global Settings option and then choose "Block all sites from storing information on this". Unless you really want another way for advertisers and websites to track your online movements there is little need for this option to be enabled. You may want to check your video and microphone options as well.

Even after all this the Demand 5 flash videos were still having problems playing and although the options I just disabled are worth doing anyway to prevent online tracking by advertisers it didn't fix the Demand 5 TV catch up problem.

Flash is a well known CPU killer on websites and I have seen whole PC's crash due to one to many flash object on a web page being left open too long. Just open a page with a few flash movies and watch the CPU rise in your task manager if you want proof - Chrome seems particularly bad for this but despite this many sites continue to try and write their whole website in Flash which is a BAD IDEA!

However I did notice that the page that held the movie also contained a large number of other banner adverts that used both flash and animated gifs and I wondered whether there was some sort of problem occurring due to all the techniques advertisers now use to try and overcome advert blockers.

Basically websites or plugins use a timer to re-insert banners that have been removed from the HTML DOM by advert blocking plugins like AdBlocker which also uses a timer to remove any adverts re-inserted this way. Both sets of code doing the opposite to each other every split second - not good for performance!

Because both the advertiser and blocker are using the same methods to scan and modify the DOM constantly it basically turns it into one big performance nightmare in which your DOM is constantly being re-written on the fly. The less client code that runs the better KISS IT (Keep It Simple Stupid) for a multitude of reasons.

The Fix for the Demand 5 website Performance Problem

I opened Firefox and went to install a plugin I use on my other computer called Flash Blocker but I noticed a new plugin had been released called Image and Flash Blocker 0.7. I thought I would try it out.

At first I thought something hadn't loaded as I couldn't see any options under my Tools menu however you need to use the context menu (right click the mouse button) to use the add-on and see the various options.

Choose "Image and Flash Blocker" from the context menu and then select "Images off, Flash off" and hey presto most of the imagery, banner adverts, flash banners and movies disappear.In fact most of the Demand 5 page is blank without these features enabled.

The flash movies are replaced with a little red circle with a white "F" (for Flash) in the middle and if you want that particular flash movie to appear you just click it.

I turned every image and flash movie off on the latest episode of the show I wanted to watch e.g Burn Notice, NCIS, Archer, The Mentalist etc, and then I clicked the main movie screen to turn the video back on.

When the adverts had finished playing the TV show played without a single stall, stoppage or flicker. Hey presto problem solved! Big slap on the back for moi.

Remember - too much whizz bangery can cause performance issues

Remember images, Applets, Active-x objects, Flash and all the other fancy whizz bangery that comes with HTML 5, CSS 3 and modern JavaScript libraries is all well and good but more often that not it can cause a massive performance overhead on your computer. The best tactic is to turn it all off and then only turn on what you need once you know you need it and the clients browser supports it.

In the case of fixing Demand 5 catch-up TV it was definitely a case of less the better as it seemed that too much was going on behind the scenes to allow their TV shows to play smoothly. What exactly is happening I don't know without access to their source code but this is definitely a workaround for the Demand 5 performance problem that works!

Hopefully this blog article will help others overcome the same problem. I have tried writing comments with a link to this article on most of the shows I watch as they all contain similar complaints but for some reason they don't like the fact I put a link into my comment or try to help people solve their own technical issues.

So if you watch Demand 5 and have performance problems remember - turn off images and flash and then turn on the only flash movie you need - the TV programme you are trying to watch and then enjoy it without any flickering or stopping every 20 seconds.

Tweet