Changing Blogger With JavaScript

Using JavaScript To Edit Blogger Layout

By Strictly-Software

If you are using Googles blogger to output content then there will often be times where you want to edit the layout depending on the page you are on, or the content that is shown.

I recently had a site where I needed to edit the content in multiple places depending on whether a GoFundMe post was being shown on the homepage, or it's own page. As I also had a mini GoFundMe logo in the sidebar I didn't want both images to be shown at the same time.

I also had a Custom Google Search Bar which only searched a certain number of websites related to the content on the blog.

Therefore the logic I needed was to.

1. Once the page had loaded I needed to ensure if a post of GoFundMe was shown on the page then I needed to remove the sidebar image in ID #Image4.

2. Just doing a test for #Image4 doesn't work as the widgets change names server side before output therefore I need to list out the pages / URLS that the post appears on where I know that the image in the side bar and the main post image both exist.

3. I need to add text to a special Google Search Bar when the site has loaded, sometimes this can be slow depending on Google's Servers, so therefore I decided to put it in a window.onload event to ensure all other images and external items had loaded before I edited the input bar.

4. On focus of the search bar I need to clear it to allow people to put their own search terms.

5. On leaving the search bar if it's empty then I need to put the default text back in, a simple onblur, onfocus toggle.

As the widget I added (an HTML/JavaScript widget), was at the bottom of the page the likelihood of the code running after these items had loaded was high so the chances of the code running before the search bar being loaded was low. 

However on the chance it did run, the window.onload event unlike DOMLoad events ensure that all external items have loaded. If I tried a DOMLoad event there was a small chance that the code would run before the items it was affecting had loaded.

Therefore to test that the page was showing a post with the GoFundMe on it and therefore removing a server side loaded item from the page I used Regular Expressions. 

I looked at all the places the pages loaded this post and they were the homepage, the homepage for mobile or non mobile sites, with or without the Google # link for the search bar tab being in the URL, plus certain labels that only this post appeared on.

The Regular Expression I came up with that worked using my test page I made in HTML & JS that worked was this.

^https:\/\/sitename.blogspot.com\/?(\??(m=\d)?(#gsc.tab=\d+)?|\S+?GoFundMe\S+?|\S+?\/(?:Donate|Donations|GoFundMe)\S+?)?$

This expression uses the document.location.href value to run on and is anchored from the start to end of input with ^ and $.

It always contains the main domain in the URL and I have made the ending / optional with a ? e.g escaped with a forward slash first and putting a question mark after it e.g \/?.

The next part is whether or not a querystring ? exists which is why there is an escaped question mark first to denote a real question mark, followed by another one to say it "May or may not be there". 

The next part is a check to see if there has been a toggle used on the site to change from a mobile view to a website view. They use a simple querystring value for this m=0 (website), m=1 (mobile), so I have (m=\d) which denotes m=(any digit), just incase they change or append digits.

This is the 1st part of an OR section wrapped in a group. I could have put ?: at the start of the group to say "do not store", but as I am not back referencing or using the group in replacements then I didn't bother.

The whole OR is broken down into this logic.

The logic contained within the OR (expression|expression|expression) is the equivalent to this long explanation:

(\??[optional back slash](m=\d)?[optional mobile/website URL toggle to any digit](#gsc.tab=\d+)[optional anchor that denotes Google has loaded up its search bar code as if you are quick you can catch a document.location.href value without this on it]. Then a wrap around the main post called GoFundMe with \S+? which makes it look for non space characters 1 or more times before or after the words GoFundMe which appear on the posts page in the middle of the URL e.g https://sitename.blogspot.com/2021/10/httpssitename.blogspot.comGoFundMe.html.html?m=0#gsc.tab=0 and then finally a look for the 3 labels that are used to find the page and appear in the URL like so https://sitename.blogspot.com/search/label/Donations?m=0 (as you can see when I copied the URL the page hadn't loaded in the Google Search Bar which is why there is no #gsc.tab=0 in the URL) so this is why I have made that part optional as well.)

So that regular expression handles the following URL's whether or not the Google code has loaded or there is code to force it show a mobile site or website version of the blog.

https://sitename.blogspot.com

https://sitename.blogspot.com/

https://sitename.blogspot.com/?

https://sitename.blogspot.com/?m=0 or https://sitename.blogspot.com/?m=1 

https://sitename.blogspot.com/#gsc.tab=0

https://sitename.blogspot.com/?m=1#gsc.tab=0

https://sitename.blogspot.com/2021/10/httpssitename.blogspot.comGoFundMe.html.html#gsc.tab=0

I made a quick function call to querySelector so that it is short, I always used G when it was document.getElementById(val), so I used it again as to me it means GET G(x) = GetMe(x) a reference to the small framework I made which is on the blog somewhere e.g

G = function(v){return document.querySelector(v)};

Then I can reference objects by class, selector or ID easily with a one letter function e.g

var el=G(".gsc-search-button > input");

G("#Image4").remove();

The final code I inserted into the JavaScript/HTML widget was this.

<script type="text/javascript">
G = function(v){return document.querySelector(v)};
window.addEventListener("load", function() 
{
	if(G("#gsc-i-id1")){
		G("#gsc-i-id1").value = "Search for similar articles here";
	}
	G("#gsc-i-id1").addEventListener("focus",function(){
		G("#gsc-i-id1").value="";
	});
	G("#gsc-i-id1").addEventListener("blur",function(){
		if(G("#gsc-i-id1").value==""){
			G("#gsc-i-id1").value = "Search for similar articles here";
		}
	});	
});

// As these elements are loaded server side they will exist when any JavaScript gets to run so no need to put inside the Window.Onload event
var exp = new RegExp(/^https:\/\/sitename.blogspot.com\/?(\??(m=\d+)?(#gsc.tab=\d+)?|\S+?GoFundMe\S+?|\S+?\/(?:Donate|Donations|GoFundMe)\S+?)?$/,"gi");
if(exp.test(document.location.href)){
	G("#Image4").remove();
}

// search button never appears wide enough to say "Search" so expand
var el=G(".gsc-search-button > input"); // Use selector to get child input of class gsc-search-button which is used numerous times

if(el){el.style.width="60px";} // just a safety test to ensure it's there - which it should be
</script>

It works and I am happy to see no double GoFundMe images on the page and the focus/blur code working fine.

It is just an example of using JavaScript to modify the DOM when you cannot use Server Side code to initially output it.

© 2021 By Strictly-Software

Making A Super Trim Function

Using Regular Expressions To Make a SuperTrim() Function

By Strictly-Software

How many times has there been when you have two bits of text that you have extracted from various websites, or feeds or even databases and tried to compare them but they would not match?

I know I wrote a little example of when two different ASCII space characters are used within SQL the other day and how to check and remove them to make a match but what about all the various ways you can HTML Encode spaces like     and   plus others that make up a CrLF or just a Cr or Lf, a bit like the VbCrLf constant for a carriage return and line feed, either using Environment.NewLine or constants that hold values for \r \n and also maybe a tab \t.

All these are spaces that need removing and with a special function that uses regular expressions they can all easily be removed .

I use this function in MS SQL with a CLR C# UDF as well as Extending C# projects with a new SuperTrim() method like so:

public static string SuperTrim(this string value)
{
    string newval = "";

    // match each type of space from start of input up to a word character that may or may not have spaces in between
    // e.g a sentence like Hello There John and then removes the same space characters to the right to the end of sentence.
    string re = @"(^(?: | | |\s|\t|\r|\n)+?)(\w+[\s\S]+?\w+)((?: | | |\s|\t|\r|\n)+?$)";
    Regex regex = new Regex(re, RegexOptions.Compiled | RegexOptions.IgnoreCase);

    newval = regex.Replace(value, ""); // replace each space HTML char with nothing

    return newval;
}

It is pretty easy enough to create yourself a test page in HTML using JavaScript with a couple of textarea input boxes for the test value containing encoded spaces a button to run a JS function that runs the regex as seen in the C# example and then outputs the result in another box. 

The regular expression is interchangeable between languages, that's what I love about Regular Expressions, they can be tested and played about with on a simple HTML page with JavaScript and then one the expression works you can easily move it into whatever language you are working in e.g C# or PHP.

For example this encoded text:

      Rob Reid       

Then after running the regular expression or string newValue = EncodedValue.SuperTrim() method in C# or JavaScript you should get this value with no encoded characters left.

Rob Reid

I find extending whatever language I am writing in to include a SuperTrim() function very handy. If you were handling URL's you might want to remove %20 and the + sign, you can always add more or less into the expression depending on your needs of course like values for nulls or \v for vertical tabs depending on the content you are handling.

By Strictly-Software

© 2021 Strictly-Software

A little piece of SQL and a Regular Expression that saves me a hell of time

Betting, SQL and Regular Expressions

By Strictly-Software

Following on from the previous post, now that I am a professional trader on Betfair, or rather a bettor with some good luck, I managed to pick the 33/1 winner of the Derby out on Saturday somehow, I am always working on my code and my Automatic betting BOT which can TRADE for me on the Betfair Exchange.

However as I also run a Facebook page where I post up free tips from myself, and international tipster friend and other "pro" tipsters > https://www.facebook.com/ukhorseracingtipster/ I have to work on my social media quite a lot.

One of the things I had to do every night was get up at 12am to set my BOT running to import the days race cards, runners, prices and related statistics that help my systems pick out horses to bet on, and therefore to post up to tip.

However one of the most annoying jobs I found myself repeating was every night having to go to SportingLife.com and the daily race card page, copying out UK and Irish races, reformatting it and then adding it in to one of the 1st posts of the day.

Now I have tweaked the import process so I can import races from tomorrow, or 2 days away whenever I like rather than waiting until midnight just to get the current days races. I can now insert race cards for days ahead whenever I like and just concentrate on the betting and tipping. Running the BOT just to get up to date prices etc.

The post would always start like this....

Monday's racing comes from Gowran Park, Leicester, Lingfield Park, Listowel, Pontefract and Windsor.

Which means re-ordering the copied text into alphabetically listed race courses, adding in the day of the week and of course changing the last course name so it doesn't precede a comma but the word "and x" and have a full stop after it.

Not much work you might think to yourself. Well no not really, fun with refreshing pages that don't have data and are stuck in some cyclical mode, but it can take up to 20 minutes.

Therefore today I wrote a small piece of SQL that can work on any days data already imported to format this sentence correctly. I could have done it without a regular expression but then where would the fun be in that?

So here is the code:

DECLARE @Courses varchar(255),@Message varchar(255), @day varchar(15), @racedate date, @Days int
SELECT	@Days = 0 -- how many days into the future do you want data for - put 0 for today
SELECT  @Racedate = CAST(DATEADD(DAY,@Days,GETDATE()) as DATE) -- convert GETDATE() (todays datetime) into a DATE format
SELECT  @Day = DATENAME(weekday, @Racedate) -- get the current weekday for the date above
SELECT	@Courses = COALESCE(@Courses + ', ',' ') +  CourseName -- build up a CSV of courses
FROM	RACES as ra with (nolock)
JOIN	COURSES as c with (nolock)
	ON	ra.CourseFK=c.CoursePK
WHERE	Abandoned=0
	AND Racedate = @Racedate
GROUP BY CourseName
ORDER BY CourseName 
-- get the last course in the list, I could have used a UDF with comma counting but that would mean replicating the above code, easier in a regex
SELECT @Courses = [dbo].[udf_SQLRegExReplace]('(^[\s\S]+?)(?:, ([^,$]+?$))',@Courses,'$1 and $2')
SELECT @Message = @Day + '''s racing comes from ' + LTRIM(@Courses) + '.'
SELECT @Message

If you want to know what the regular expression below does:

SELECT @Courses = [dbo].[udf_SQLRegExReplace]('(^[\s\S]+?)(?:, ([^,$]+?$))',@Courses,'$1 and $2')

Then watch out for hopefully a soon coming Amazon book I am writing on getting started with regular expressions I know there are loads out there but for me personally I learn by doing things not reading things and this is what customer type the book is aimed at. 

It gets you writing regular expressions straight away and then working out what they do rather than just listing all the possible flags, identifiers and other boring stuff that a lot of reference books contain.

If you have a quicker RegEx for doing the changing of the last word etc then please post it in the comment section. I am always willing to look at other ways of doing things.

If you are into betting then always check out my facebook page for UK, Irish, French and International tips > https://www.facebook.com/ukhorseracingtipster/

By Strictly-Software

© 2021 Strictly-Software

Debugging Regular Expressions

Regular Expressions - A guide to debugging

Regular expressions are a powerful tool to master and a great skill to learn as a good expression can save many lines of procedural code. They can be used for many tasks and are especially great for validating user input such as form validation as well as great for cleaning content such as HTML entered from a content management system. When used with remote content grabbers they are great for extracting sections of text from larger pieces as well as useful for quickly reformatting it.

Although a useful and powerful tool when used correctly its also quite easy to use them either accidentally or on purpose for destructive purposes. For example a sure sign that a regular expression has gone haywire and needs looking at is when the CPU on the computer running the expression suddenly spikes to 100%.

This can often happen for numerous reasons but a couple of the most common are when the expression gets itself into an internal loop because of a partial match that then matches multiple sections of text, a complicated pattern that matches nothing or because of a negative lookahead. All of these problems may not be spotted if the text you are matching against is quite small but when the text is quite big the problem amplifies. In fact some people have designed complicated patterns that are specifically created to match nothing but consume CPU whilst doing so. For more information about the dangers of regular expressions and SQL Denial of Service Attacks read the following two articles.

http://blog.strictly-software.com/2008/10/sql-denial-of-service-attacks.html

http://blog.strictly-software.com/2008/10/dangers-of-pattern-matching.html

Common Problems

Sometimes trying to work out what complex regular expressions are doing can be quite hard especially if you didn't write it in the first place. For a novice looking at an expression like the following it could be quite confusing trying to work out what is going on:

var re_getme = /^<([^> ]+)[^>]*>(?:.|\n)+?<\/\1>$|^(\#?([-\w]+)|\.(\w[-\w]+))$/i

Therefore when you get stuck with a complex expression its a good idea to break it down into tiny little steps and build it up bit by bit. If your pattern isn't matching what you expect then comment it out and start a new expression. Start with a cut down simple version of your pattern and get it matching or replacing something and then build it up from there.

Even for performance reasons it might be worthwhile considering breaking a complicated expression down into multiple expressions. If you are using a pattern to replace certain content then using multiple expressions is a good idea especially if your existing expression contains lots of OR statements or negative matches.

Multiple ways to skin a cat

The great thing about regular expressions however is that they offer you the ability to handle a task in multiple ways. For example both of the following expressions look for HTML tags:

var reHTML1 = /<[^>]+?>/;

var reHTML2 = /<(.|\n)+?>/;

The first one matches an open bracket and then uses a negative match to look for any character apart from the closing tag one or more times before then matching the closing tag. The second one matches an open bracket and then any character or any space one or more times and then a close tag.

Therefore if you are getting stuck and have spent a lot of time trying to crack an expression one way but its not just working try to attack it from a different angle.

Negative Matching

Trying to do negative matching can be quite complicated especially if you trying to match a string rather than one character or a group of characters. For example this negative expression just removes any characters that are not between A to Z or 1 to 9.

[^A-Z1-9]+

Because this expression doesn't require any ordering of the characters its not matching its not hard for the engine to carry out. The following expression is a bit more complicated as its doing a negative lookahead to make sure the text is not one of the specified strings in the OR list.

(?!https?://.*(webmail|e?mail|live|inbox|outbox|junk|sent)).*

Expressions that carry out negative lookaheads are okay as long as the text being searched is not too large however trying to do complex lookaheads with large pieces of text is a sure way of maxing out your CPU.

Using Placeholders

If you are trying to carry out replacements of text that involve a negative match then you may want to consider using a placeholder to make your code simpler and perform better. Instead of trying to negatively match a string that you don't want to replace you do a positive match on this string replacing it with a placeholder then do your replacement before putting the value back in.

For example in my HTMLEncoder object I have a method that HTML Encodes text passed to it. When it comes to replacing ampersands with entities I cannot do a straight replacement as I want to keep any ampersands already in the text that are there as part of an existing numerical or HTML entity as I cannot guarantee the text I am encoding doesn't already contain some encoded characters. Therefore instead of trying to do a negative match for &# I do a positive match replacing that string with a placeholder. I then do my replacement of all other & ampersands before putting back my placeholder values.

// replace ampersands from existing numerical entities with a placeholder
s = s.replace(/&#/g,"##AMPHASH##");

// replace all remaining ampersands with their entity version
s = s.replace(/&/g, "&");

// put back in my placeholder values for numeric entities
s = s.replace(/##AMPHASH##/g,"&#");

Re-use objects and Cache Expressions

If you using a regular expression object and carrying out multiple expressions or executing the same expression multiple times then you should look into caching the expression if possible. If you are using wrapper functions to carry out certain tasks then try not to create a new regular expression object on each call to the function. The creation and destruction of a COM object can be very expensive so try to use global objects that are created once and used by all your functions and then destroyed at the bottom of your page or script.

Differences in Regular Expression Engine

Although most languages have inbuilt support for regular expressions the quality of their engines differ as well as the syntax of the expressions themselves. I have found Microsoft's Visual Basic Regular Expression Engine is very poor and doing complex expressions in VBScript or ASP Classic can often cause CPU spikes on the web server. I have also seen certain patterns that work in other languages cause problems with this engine when the text being matched contained Unicode characters.

The JScript engine is a lot better than the VBScript engine and if you are using ASP classic then I would recommend using server-side JScript for any complex regular expressions. Not only is the engine better you will have much more flexibility with your code such as using lambda expressions and being able to pass in a function as the argument for the replacement value.

In SQL Server there is no inbuilt regular expression function like there is in MySQL but you can use LIKE or PATINDEX for basic patterns or if you are using SQL 2005 - 2008 you can extend the CLR and build some user defined functions that hook into .NET to allow you to carry out proper regular expression functions. In SQL 2000 you can use OLE and the built in extended stored procedures sp_OACreate, sp_OAsetProperty and sp_OAMethod to instantiate the VBScript.RegExp COM object to carry out regular expression tasks. However this method can be quite an overhead and should be used carefully if required especially if the create and destroy methods are wrapped in a UDF as it will mean a new COM object is created every time the function is called and we all know object re-use is a key ingredient when it comes to performance especially with COM objects.

Knowing when you have a problem

If your regular expression code is client side Javascript or VBScript then a good indicator that you have a problem is when your browser hangs or pops up an unresponsive pop-up. Check your task manager and investigate the CPU usage as a long running expression that is killing your machine will surely be maxing out your PC's CPU.

If the code is server-side either in a database or web server then if you are viewing your servers performance monitor and you notice the CPU jump up in blocks of 25% (on a quad), or 50% (on a dual) or 100% (single), stay there for a few seconds and then drop down again then the chances are its regular expressions doing the maxing.

Regular Expression functions for reformatting content for Wordpress

Using RegEx to reformat Wordpress content

If you read my blog you will know I am "Mr Automate" and I specialise in scraping, reformatting, spinning and automatically posting content to Wordpress, Twitter and other sites.

I use Wordpress for a couple of my blogs and because the well known plugin WP-o-Matic has stopped being supported and the only replacements are paid for plugins like WP Robot I have basically written my own custom version to handle my automated content imports.

A couple of useful Regular Expressions you might like to use which I find particular helpful are below.

I am using PHP as that is the language Wordpress is written in but the core Regular Expressions should be easily transferable between languages.

1. Removing IFRAMES

As YouTube has moved from it's old OBJECT / EMBED nested tag soup to an IFRAME many feeds containing videos will now contain IFRAME's which need to be handled correctly as we all know that viruses can be transfered through rouge IFRAME content.

If you are just importing content from a feed without parsing and checking it then just imagine if for whatever reason the feed you are using one day contains an IFRAME with a SRC pointing to a dodgy virus infected URL. If you don't sanitize the content you will be inserting this dangerous content into your own blog for your visitors to become infected by.

Although most modern browsers and virus checkers are good at picking up on potential malicious URL access it is always wise to run your own checks using a white list of "allowed" domains rather than a "black list" of banned ones.

This code is a basic example of looping through all the IFRAMES on a page, checking their SRC tag for a known white list of acceptable domains and then replacing any IFRAME's that don't match.

// match all XHTML IFRAME tags where $content holds your HTML
$videocount = preg_match_all("@(<iframe[\s\S]+?<\/iframe>)@",$content,$videos,PREG_SET_ORDER);

foreach($videos as $video){

 $object = $video[1];

 // only allow certain domains e.g youtube, dailymotion, vimeo, cnn, fox, msnbc and the bbc
 if(!preg_match("@src=['\"]http:\/\/(www\.)?(?:youtube|dailymotion|vimeo|cnn|fox|msnbc|bbc)\.@i",$object)){
  
  // replace the IFRAME as we don't know where it is pointing to

  $content = preg_replace("@" . preg_quote($object) . "@","",$content);
 }
 
}

2. Re-sizing images

I have already written an article about how you can use regular expressions and a callback function to reformat videos or images so that they fit your template and this follows that theme.

PHP like Javascript and other decent languages offers the ability to create anonymous (unnamed) functions on the fly to be used as callbacks in regular expression replace functions.

Passing a function as a parameter within another function is known as a lambda function and this is very useful in certain situations such as parsing HTML where you want to remove illegal tags and attributes.

This example is pretty niche to my own needs but the idea is useful and can easily be converted to your own reformatting needs.

Basically I show little Google Adsense box adverts at the top right of all my articles and as these are floated to the right I don't want any images within my articles that are less than the size of my content area (minus the size of the advert) to also be aligned to the right as this causes a horrible looking mess.

Therefore I use this code to ensure that the Wordpress class "alignleft" is always inserted on the first image in any article but only if the width of the image is less than 350px. It doesn't matter if the image isn't right at the top of the article (although they usually are) but it helps to ensure a smooth flow and layout.

The Preg_Replace function in PHP is nice in that you can limit the number of replacements to one or more if you so wish and in this case I only want to reformat the first image. You will see that the lambda function I use also contains multiple other regular expressions and there is nothing stopping you from writing complex code within these anonymous functions.

$content = preg_replace_callback("@(<img[^>]+?class=['\"])([\s\S]*?)(['\"][\s\S]*?>)@i",
  create_function(
  '$matches',
  'preg_match("@width=[\'\"](\d+)[\'\"]@",$matches[0],$widthmatch);
  $chk=true;
  if($widthmatch){       
   if(is_numeric($widthmatch[1])){
    if($widthmatch[1] >= 350){         
     $chk = false;
    }       
   }
  }
  if($chk && !preg_match("@alignleft@",$matches[2])){
   $res = $matches[1] . trim( preg_replace("@alignright@","",$matches[2]) . " alignleft") .$matches[3];
  }else{
   $res = $matches[1] . preg_replace("@alignright@","",$matches[2]) .$matches[3];
  }
  return $res;'),$content,1);

As you can see the lambda anonymous function carries out the following logic

• It looks for the first image that has a class attribute on it and passes the match collection into the anonymous function.
• It then checks the image for a width attribute using another regular expression.
• If found it then checks that the value of the width attribute is numeric and if so it tests to see if the value is over my allowed limit of 350px setting a flag if so.
• I then use another regular expression to test for the existence of the standard Wordpress class "alignleft" and if it isn't found then I add it in making sure to replace any "alignright" class if it exists.
• The lambda function then returns the new reformatted content of the image which is then used as the replacement value in the Preg_Replace function.

Just two useful examples which can be expanded upon by anyone interested in Regular Expressions and AutoBlogging.

Remember if you require any custom plugin work for Wordpress, including special reformatting functions, scraping code or content automation then contact me for a quote.

HTML Super Trim Functions

Remove non breaking spaces and other white space either side of strings

Most languages have a Trim function that removes white space from either side of a string.

However a lot of the time these Trim functions will only remove standard white space e.g if you hit the space bar a couple of times and not other forms of white space such as tabs, new lines or HTML space characters such as non breaking spaces whether they are HTML entity encoded: or Numerically encoded e.g

Therefore sometimes you may need a "Super Trim" function that will handle the removal of all types of space characters including HTML entities.

The following have been written in PHP but can easily be converted into C# or VB. The main part to take away is the regular expression used within each function which replaces a string containing one or more space characters whether they be control characters or HTML entities from either side of the string.

// wrapper function to do trim both sides
function HTMLTrim($text){

 // call both functions at once
 return HTMLLeftTrim(HTMLRightTrim($text));
}

// removes spaces and   at the beginning of strings
function HTMLLeftTrim($text){
 
 // remove space to the left of the text
 return preg_replace("@^( | |\s)+(\S+)@","$2",$text); 
 
}

// removes spaces and   at the beginning of strings
function HTMLRightTrim($text){
 
 // remove space to the right of the text
 return preg_replace("@(\S+)( | |\s)+$@","$1",$text);   
 
}

You can test this out in a simple PHP page with the following code:

$str = "     hello there        ";

echo "before trim its '" . $str . "'";

echo "<br><br>now its '" . HTMLTrim($str) . "'";

Which returns the following output:

before trim its '     hello there      '

now its 'hello there'

I find it very useful when I am scraping content from the web and need to handle the removal of a mixture of standard spaces and HTML spaces.

PHP for obtaining the follower count of a Twitter Account

Get Twitter Follower Count using Regular Expressions

I came across this bit of PHP code the other day which is aimed at getting the follower count of a Twitter user.

It seems like overkill to me and is a mixture of regular expressions, string parsing, callback functions and a lot of head scratching.

The user obviously knows that the follower count HAS to reside within an element within the DOM with the id of follower_count so why not just use one single regular expression to target that element and return it's guts instead of all the DOM loading, callbacks and string parsing?

I might be missing something that someone could tell me but this seemed like a long way to go about a simple scrape job.

// Get the number of twitter followers

function string_getInsertedString($long_string,$short_string,$is_html=false){
if($short_string>=strlen($long_string))return false;
$insertion_length=strlen($long_string)-strlen($short_string);
for($i=0;$i<strlen($short_string);++$i){
if($long_string[$i]!=$short_string[$i])break;
}
$inserted_string=substr($long_string,$i,$insertion_length);
if($is_html && $inserted_string[$insertion_length-1]=='<'){
$inserted_string='<'.substr($inserted_string,0,$insertion_length-1);
}
return $inserted_string;
}

function DOMElement_getOuterHTML($document,$element){
$html=$document->saveHTML();
$element->parentNode->removeChild($element);
$html2=$document->saveHTML();
return string_getInsertedString($html,$html2,true);
}

function getFollowers($username){
$x = file_get_contents("http://twitter.com/".$username);
$doc = new DomDocument;
@$doc->loadHTML($x);
$ele = $doc->getElementById('follower_count');
$innerHTML=preg_replace('/^<[^>]*>(.*)<[^>]*>$/',"\\1",DOMElement_getOuterHTML($doc,$ele));
return $innerHTML;
}


// To display it

<?php echo getFollowers("username"); ?>

Here is the much shorter version I wrote. It still works just as well returning the follower count of the Twitter Account username passed into it.

function getFollowers($username){
$url = "http://twitter.com/".$username;
$count=0;

$x = file_get_contents($url);

preg_match("@<([a-z][^ ]*) id=\"follower_count\"[^>]+?>([0-9,]+)\s*</\\1>@i",$x,$match);

if($match){
$count = $match[2];
}

return $count;
}

// lets check how poorly my twitter account is followed!
echo "StrictlyTweets: " . getFollowers("StrictlyTweets") . "<br><br>";

As you can see from the regular expression I am matching HTML tags (they all start with < and then a letter) and storing that tag to be used in the backtrack reference later on so that if the HTML changes from a SPAN to a DIV as long as it has the id="follower_count" with the element there will be a match.

I could have loaded up the DOM, targeted the ID and then done some regex but why bother when you can go straight for the juggular!