ETSY SHOP OPEN FOR BUSINESS!

ETSY SHOP OPEN FOR BUSINESS!

By Strictly-Software

My Etsy shop is OPEN again - if you run a WordPress site and want some tools to automate your system then check out: https://www.etsy.com/uk/shop/StrictlySoftware

I didn't know the items in my shop EXPIRED after so long so the shop was empty to viewers for the last month and a bit but now you can buy your tools from etsy or my own site for plugins: http://www.strictly-software.com/plugins (please click on some adverts and help me raise some cash)

Also my facebook page: https://www.facebook.com/strictlysoftware has information about these tools that you should read if you have purchased any of them.

It has help articles, guides on support, possible issues and fixes and much more - feel free to comment and like the page!

The basic idea behind these plugins is:

Run a site all year, 24/7 without having to do anything apart from some regular maintenance like cleaning tags that are not used very much and OPTIMIZING your database table.

So an RSS / XML feed contains your content (e.g news about something) and this goes into WordPress at scheduled times (Cron jobs or WebCrob jobs) using WordPress plugins like RSSFeeder or WP-O-Matic then as the articles are saved Strictly AutoTags adds the most relevant tags to it by using simple pattern matching like finding the most frequently used "important" words in the article e.g words in the Title, Headers, Strong tags or just Capitalised Words such as names like John Smith.

This means if John Smith became famous over night you wouldn't have to add a manual tag in for him or wait for a 3rd party plugin to add in the word to their own database so that it can be used.

Then once your article is tagged. You can choose to have the most popular tags converted into links to tag pages (pages containing other articles with the same tag) or just bold them for SEO - or do nothing.

You can set certain tags to be "TOP TAGS" which will rank them higher than all other tags. These should be tags related to your site e.g a bit like the old META Keywords.

You can also clean up old HTML, convert text tags to real clickable ones and set up a system where if a tag such as ISIS is found the tag Middle East is used instead. This is all explained on the Strictly AutoTags page on my site.

Then if you also purchase Strictly Tweet BOT PRO as well you can use those new post tags as #hashtags in your tweets and you can set your system up to either tweet to multiple twitter accounts with different formats and tags or tweet to the same account with different wording dependant on the wording in your article.



E.G if your article was about the Middle East wars you could say only post the Tweet if the article contains this word "Middle East" OR "Syria" or you could say only post if it contains the words "ISIS" AND "War".

The TweetBOT then lets you ensure the post is cached (if you are using a WordPress Caching System) by making it live first and making an HTTP request to it so it gets cached. Then it waits a custom defined number of seconds before any Tweets are sent out.

You can then specify a number of seconds between each Tweet that is sent out to prevent Twitter Rushes e.g. Where 50 BOTS all hit your site at the same time.

You can ensure no Tweets are sent out if they contain certain words, add tracking links e.g Google before the link is minimised by Bit.ly.

A simple PIN number process lets you connect your Twitter Account to your TweetBOT Account.

A dashboard keeps you informed of recent Tweets sent out, any errors from Twitter like "duplicate tweet", or if your Bit.ly account isn't working.

Plus a test button lets you test the system without sending a Tweet by taking the last post, running your settings through it such as shortening the link and post and checking all Twitter accounts are working and connected properly.

If you then link your Twitter account up to your Facebook page like I have with my Horse Racing site http://www.ukhorseracingtipster.com/ and my Twitter account @ukhorseracetips with my Facebook page facebook.com/Ukhorseracingtipster you get social media and SEO impact for free!





Check out the new live shop on Etsy for plugins and coupons if you need me to set the plugin up for your site:https://www.etsy.com/uk/shop/StrictlySoftware

You may need help due to your sites special settings or requirements so a coupon will let you help you set it up correctly for you.

Changes to Twitter API

Changes to Twitter API 1.1

As Twitter has changed their API from 1.0 to 1.1 which is totally reliant on oAuth and JSON I have had to take down the links to the Twitter Hash Tag Scanner as it was reliant on the old search RSS feeds which are no longer available.

I have had a quick look but it will take some time to re-develop and involve adding in consumer keys, access keys and so on. You would probably get blocked after a few scans anyway as you would have to login to your own twitter account to make the scans and their rate limits would apply.

Therefore I don't think a new version will be forthcoming to anyone who has purchased a previous version I can only apologise. It's a shame as I wanted to extend it but if I cannot make thousands of scans without being blocked the application just won't work with Twitters new API.

As for the Strictly TweetBot Wordpress plugin I have updated this to use the new API and I have tested it on a couple of my own blogs and it seems to be working.

Today was the switch off day so if you were using the plugin you would have noticed either:

• No tweets being sent out when you posted.
• In the Twitter message console lots of error messages saying Tweet not sent or Authentication error.

However if you upgrade to version 1.1.3 then this should fix the problem. 

You can get the latest version from Wordpress.

Also I am pissed off!

And I only just wrote a Twitter Direct Message Responder in PHP the other day which was working fine up until tonight as well!

Damn bloody Twitter.

Even with me being logged in and authenticated I was trying to get a list of my followers and for some reason I kept getting a message like this:

{"errors":[{"message":"Bad Authentication data","code":215}]}

I did write a post to the developer discussion boards on Twitter but as always I have cracked the problem before I got a response.

Basically I am using a very common Twitter / oAuth class which is used by my Twitter Plugin and many other plugins use it as well.

To fix the problem I had to do the following:

Change line 29 in the Twitter class to:

 /* Set up the API root URL. */

 public $host = "https://api.twitter.com/1.1/";

This resolved the issues in my own wordpress plugins which solved sending normal tweets out but to get my Direct Message Responder code working I needed to do one more thing.

Whereras before I was making use of a simple file_get_contents call to an XML feed which Twitter has now abandoned for JSON I had to change this to use the inbuilt HTTP request functions in the Twitter class e.g

$response = $oauth->get($followers_url);

This returns 20 of your new followers (I have not worked out how to get more yet) but in a JSON object.

You can either loop through the nested objects of you could use json_encode to convert the object to a string to do a simple regex to just get a list of screen_names e.g

$body = json_encode($response);

preg_match_all('@"screen_name":"([\s\S]+?)",@i',$body,$matches,PREG_SET_ORDER);

And that solved the problem!

Using the Wordpress RSS Feed Object and handling Timeouts

Handling timeouts with the Wordpress RSS Feed Object and SimplePie

In later versions of WordPress they have implemented their own RSS feed object which is basically just a wrapper for the well known RSS reader plugin SimpliePie.

On some of my own WordPress websites I need to make use of some RSS feeds to show results from various places such as horse racing results.

I first try and obtain an RSS results feed from Betfair and if that fails I then resort to a backup from the BHA.

However sometimes the first feed loads rather slowly and I end up with the lower quality feed due to the WordPress object timing out after 10000 milliseconds.

Even though I cache the results I require the first load of the feed to take however long it needs to - up to 30 seconds if it needs the time but due to the 10 second timeout I often don't get the first feed loaded into my cache.

Even after looking at the WordPress RSS feed object there seemed to be no easy way to pass in a timeout parameter into an overloaded constructor therefore the feed was always timing out after 10 seconds.

For all I know there is a way to set the HTTP load timeout with the WordPress RSS object but as of yet I haven't found it plus I haven't had the time or inclination to scour the WordPress codebase looking for it.

The code I was using for my first attempt is below.

$rss = fetch_feed('http://rss.betfair.com/RSS.aspx?format=rss&sportID=7');

// Checks that the object is created correctly 
if (!is_wp_error( $rss ) ) : 
 
 // Figure out how many total items there are, but limit it to 10. 
 $maxitems = $rss->get_item_quantity(10); 

 // Build an array of all the items, starting with element 0 (first element).
 $rss_items = $rss->get_items(0, $maxitems); 
endif;

However as you can see, although the RSS call is wrapped in an error object and I can pass in the maximum number of items I want to get. However I cannot pass in a timeout parameter.

Therefore instead of waiting for ages for a fix from anyone from WordPress I decided to jump straight over the WordPress wrapper object and go straight to the underlying SimpliePie object to get my data.

Not only does this mean no calls to wrapper objects that only ultimately point to SimplePie anyway but I have much more control over the parameters that are used for my feed collection.

I still use the 2 feeds and resort to my backup feed if I cannot get any items from my Betfair call but I now use the following code for my first attempt.

$feed = new SimplePie();
$feed->set_feed_url('http://rss.betfair.com/RSS.aspx?format=rss&sportID=7');

// set timeout to 30000 milliseconds (30 seconds)
$feed->set_timeout(30);

// set number of items to return to 10
$feed->set_item_limit(10);
$feed->set_stupidly_fast(true);

// enable caching and set the duration to cache for in seconds (200 seconds)
$feed->enable_cache(true);  
$feed->set_cache_duration(200);
$feed->init();
$feed->handle_content_type();

if (!$feed->error()):

 // get up to 10 items
 $rss_items = $feed->get_items(0, 10); 

 // if there are less than 10 items get the actual number in case I need to know
 $maxitems = count($rss_items);
else:
 // error output it
 echo "Error is " . $feed->error();
endif;

As you can see the ELSE branch just outputs an error message and this is just an example of how you can access the error if you need to. I don't use it on my site but it is useful to know if you are not using the WordPress error object.

The HTTP load timeout is easily set with the following line of code:

$feed->set_timeout(30);

Since I have changed my first attempt at loading the feed to skip the WordPress wrapper object with a call to SimplePie I haven't had any problems at all.

You might find this helpful if you are doing your own work with RSS feeds and WordPress.

Twitter changes their format for status feeds

Tweets not showing up? Twitter changes their feed format for status feeds

You may have noticed over the couple of weeks that the usual Tweets from my Twitter account @strictlytweets have not been showing in the right hand side bar.

This is because Twitter have changed the URL that you must request to get those tweets from.

http://twitter.com/statuses/user_timeline/strictlytweets.json?count=10

To their new API which uses a secure URL (e.g from http to https)

https://api.twitter.com/1/statuses/user_timeline.json?screen_name=strictlytweets&count=10

Now if you have a callback function it would be called in the same way with a parameter passed in the URL for the name of the function to be called.

Although I am not sure whether Twitter have changed their name of their standard function which formats Tweets into links from twitterCallback to twitterCallback2 if you take a look at the one blogger uses you can see they are calling it twitterCallback2.

You can see their script here:

http://twitter.com/javascripts/blogger.js

Therefore if you have a special function that you need to call to format your tweets in your own style, cache on your own server or scroll them you will still need to add the name of your callback function to the script source and make sure it accepts a JSON hash table of Tweet objects.

You will also need to reference this script before calling the new script URL that gets the statuses.

If you look at this blog you can see that the first script loads in bloggers own formatting of the Tweets in  through a call to blogger.js and the second one gets the new Twitter feed status format in the new URL format using JSON.

<script type="text/javascript" src="http://twitter.com/javascripts/blogger.js"></script>
<script type="text/javascript" src="https://api.twitter.com/1/statuses/user_timeline.json?screen_name=strictlytweets&callback=twitterCallback2&count=6"></script>

Twitter are constantly changing and like most sites are moving to secure URL's e.g https and using JSON instead of XML/RSS to return data to the user.

So if you have a Twitter account and want to show your own Tweets on your site and it hasn't been working lately this is the reason why.

Just switch over to the new https format and you will be okay as long as you also implement some form of caching to handle the rate limit which is only 150 requests per hour per IP address NOT per domain. Or if you authenticate each request with OAuth you can make 350 requests per hour.

As my IP address is also shared by a lot of other sites that use this feature the Tweets may still not appear until I come up with a form of caching (which I haven't got round to doing) but the logic would be something like this.

• Make the request to the new API URL.
• Format the HTML using your callback function.
• Make an AJAX request to a server side script (PHP/ASP/.NET etc) that stores the formatted HTML as a text/html file.
• If you get back a 400 Status Code (Bad Request) which is the code Twitter send you when you exceed their rate limit. Then you can display this HTML file instead.

You can read more about rate limits here.

Error copying tables importing or exporting the Geography or Geometry data type in SQL 2008

Error importing and exporting the Geometry and Geography data types in SQL 2008

Today I had to make some work live that involved copying a database containing UK Postcodes and their related geo-location data to another SQL 2008 server.

The table contained a list of all UK Postcodes as well as their longitude and latitude and a GeoLocation column that was based on the new SQL 2008 DataType Geography.

However when I tried to use the Import/Export wizard to copy the data I got to the Review Data Type Mapping page and was met with the following error message.


Found 1 unknown column type conversion(s) You are only allowed to save the package.




Viewing the full error details revealed that the problem was down to SQL not understanding the Geography data type in the table I wanted to export.


The full error message details are below:

[Source Information]
Source Location : (local)
Source Provider : SQLNCLI10
Table: [dbo].[Country]
Column: CountryGeography
Column Type: geography
SSIS Type: (Type unknown ...)
Mapping file (to SSIS type): C:\Program Files\Microsoft SQL Server\100\DTS\MappingFiles\MSSQLToSSIS10.XML

[Destination Information]
Destination Location : (local)
Destination Provider : SQLNCLI10
Table: [dbo].[Country]
Column: CountryGeography
Column Type: geography
SSIS Type: (Type unknown ...)
Mapping file (to SSIS type): C:\Program Files\Microsoft SQL Server\100\DTS\MappingFiles\MSSQLToSSIS10.XML

[Conversion Steps]
Conversion unknown ...
SSIS conversion file: C:\Program Files\Microsoft SQL Server\100\DTS\binn\DtwTypeConversion.xml

First of all I checked that the database compatibility mode was set to 100 (SQL 2008) and not 90 (SQL 2005) and once I had confirmed both databases were the correct format I checked the mapping conversion XML file on the server I was doing the import from.

This XML mapping file is located at the following path C:\Program Files\Microsoft SQL Server\100\DTS\MappingFiles\MSSQLToSSIS10.XML.

On opening the file I could see that there was no mention of the geography OR geometry data types which explained why the DTS package wizard could not carry out the operation.

To fix this I copied one of the other similar data types (varbinary) and re-inserted it into the file twice before changing the names to Geography and Geometry.

You can just copy and paste the following XML into the file.

  <!-- geography -->
  <dtm:DataTypeMapping >
    <dtm:SourceDataType>
      <dtm:DataTypeName>geography</dtm:DataTypeName>
    </dtm:SourceDataType>
    <dtm:DestinationDataType>
      <dtm:SimpleType>
        <dtm:DataTypeName>DT_IMAGE</dtm:DataTypeName>
      </dtm:SimpleType>
    </dtm:DestinationDataType>
  </dtm:DataTypeMapping>




  <!-- geometry -->
  <dtm:DataTypeMapping >
    <dtm:SourceDataType>
      <dtm:DataTypeName>geometry</dtm:DataTypeName>
    </dtm:SourceDataType>
    <dtm:DestinationDataType>
      <dtm:SimpleType>
        <dtm:DataTypeName>DT_IMAGE</dtm:DataTypeName>
      </dtm:SimpleType>
    </dtm:DestinationDataType>
  </dtm:DataTypeMapping>

Save the MSSQLToSSIS10.XML file and you should now be able to import or export a table that contains the datatypes geography and geometry.

If for some reason after editing the file it still doesn't work, try the following:

• restarting the SQL Server service on the server you are running the export/import wizard from.
• restarting your own SQL Server Client Tools.
• If you are running a 64bit version of Windows 7 like I am then you might need to also edit the same file in the 32 bit Program folder e.g C:\Program Files (x86)\Microsoft SQL Server\100\DTS\MappingFiles\MSSQLToSSIS10.XML.

My JavaScript HTML Encoder Function

Why did I write my JavaScript HTML Encoder object

One of my most popular scripts that continues to get downloaded a lot as well as spark many an email from companies and users is my HTML Encoder script that I wrote many a year back.

You can see the script in action here: www.strictly-software.com/htmlencoder

I was emailed another function that claimed to do the same work earlier and I just thought it would be interesting to other readers to know why I created the script in the first place and why many of the examples out on the net that claim to do the same job are just not up to scratch.

On many sites nowadays people are obtaining content from various sources through XML and RSS feeds and they often use JavasScript and AJAX to do part of the job. I know that when I was creating my football site www.hattrickheaven.com which was basically an exercise in using the Google AJAX API I came across the following problems:

1. Content is obtained from a multitude of sources and there is no common standard that can be relied on 100%.
2. Content is often mixed together through feed blenders like Yahoo Pipes or Blastcasta.
3. Content is often loaded in on the fly using AJAX, a Scraper Proxy to do cross domain content jacks and other formatting duties such as on the fly translations etc.
4. There is no built in function for HTML encoding in the JavaScript language.

Now I looked at many an HTML Encode solution before writing my own but the main problem I needed to overcome was that content could already be HTML encoded or partially HTML encoded and functions I came across did nothing to handle this problem.

For example if you had this bit of text you will notice that it has an encoded ampersand in the middle but the quotes are not encoded.

"Rob says hello & goodbye"

If you ran this through many of the HTML Encode functions out there including many server side ones it will double encode the & in the & so that you end up with this.

"Rob says hello & goodbye"

Now you may want this to happen but I doubt it.

Not if you want to run all your content through the same function without worrying about double encoding issues like this on your website.

This is why I wrote my HTML Encoder object as it puts into practise the technique I used whilst writing ASP classic sites that supported the UTF-8 character set.

With multi-lingual sites that display Arabic or Japanese character sets you don't want to be using the inbuilt Server.HTMLEncode function on all your textual input as it will triple the size of everything you store by converting every non ASCII character into a &#XXXX; encoded string.

When your database is set up to correctly store your text as Unicode (nvarchar etc) and you are outputting it with the correct code-base then you need something a bit more clever to do the job as you still need to encode characters that can cause you damage such as the naughty 4 " ' < > as well as the ampersand & to make your page validate (URL's, Querystrings etc).

This is obviously for security reasons as you don't want people to be able to malform your input and break your layout or insert XSS hack vectors into your system.

However running the standard HTML Encode function will encode everything as well as cause double encoding issues. Therefore the HTML Encoding needs to be done in stages and the ampersand has to be handled correctly as it makes up part of the HTML encoded &#XXXX; format.

So whilst it comes as quite a surprise to see how popular this script is (#1 on Google for "HTML Encode with Javascript") it is not really surprising when one thinks about the problem this script solves.

Whilst I don't claim to be any kind of coding genius and I am sure the code can be improved it does do what is says on the tin and if you are thinking of using another JavaScript function to HTML Encode your content then compare it first with my tool to see if it handles the double encoding issue as if it doesn't then you might find yourself having problems down the road.

Wordpress, WP-O-Matic and custom HTML tags

Automating Wordpress posts with WP-O-Matic

If you use wordpress you should really look into the great WP-O-Matic plugin that allows you to automate postings by importing content from RSS or XML feeds. You can set up schedules to import at regular times or import on demand from the admin area.

One issue however which I have just spent ages getting to the bottom of is the use of HTML such as OBJECT and EMBED tags. As a lot of content feeds contain multimedia files nowadays and you want this content to be imported directly into your site. The problem with WP-O-Matic and Wordpress in their default mode is that you will only get this content imported when you run the import from the admin menu or the page that the CRONJOB calls directly whilst logged in as an admin or publisher.

If you try to run the page the cronjob calls e.g /wp-content/plugins/wp-o-matic/cron.php?code=XXXXX whilst logged out or allow the job to run by itself you will find that certain HTML tags and attributes are removed including OBJECT and EMBED tags.

The reason is for security to prevent XSS hacks and its possible to get round this if you require to. This took me quite a long time to get to the bottom of as I am very new to Wordpress but I managed it in the end.

1. WP-O-Matic makes use of another object called SimplePie which is a tool for extracting content from XML and RSS. This object has a number of settings for stripping out HTML and the behaviour depends on how the feed import is called.

When running the import from the admin menu a setting called set_stupidly_fast is set to true which bypasses all the normal formatting and HTML parsing. When the CRONJOB runs this is set to false so the reformatting is carried out. In reality you want to run the reformatting as it does much more than just parse the HTML such as remove excess DIV's and comment tags and ordering the results by date.

If you don't care about this formatting you need to find the fetchFeed method in the \wp-content\plugins\wp-o-matic\wpomatic.php file and force it to be false all of the time:

$feed->set_stupidly_fast(false);

If you do want to keep the benefits of the stupidly_fast function but allow OBJECT and EMBED tags then you can override the strip_htmltags property in Simplepie that defines the tags to remove. You can do this in the same fetchFeed method in the wpomatic.php file just before the init method is called by passing in an array of tags that you do want Simplepie to remove from the extracted content.

// Remove these tags from the list
$feed->strip_htmltags(array('base', 'blink', 'body', 'doctype', 'font', 'form', 'frame', 'frameset', 'html', 'iframe', 'input', 'marquee', 'meta', 'noscript', 'script', 'style'));
$feed->init();

So that takes care of the WP-O-Matic class but unfortunatley we are not done yet as Wordpress runs its own sanitisation on posts in a file called kses.php found in the wp-includes folder. If you are logged in as admin or a publisher you won't get this problem but your CRONJOB will run into it so you have two choices.

1. Comment out the hook that runs all the kses sanitisation which isn't recommended for security reasons but if you wanted to do it the following line should be commented out in the kses_init_filters function e.g

function kses_remove_filters() {
// Normal filtering.
remove_filter('pre_comment_content', 'wp_filter_kses');
remove_filter('title_save_pre', 'wp_filter_kses');

// Post filtering
// comment out the hook that sanitises the post content
//remove_filter('content_save_pre', 'wp_filter_post_kses');
remove_filter('excerpt_save_pre', 'wp_filter_post_kses');
remove_filter('content_filtered_save_pre', 'wp_filter_post_kses');
}

Commenting out this line will ensure no sanitisation is carried out on your posts whoever or whatever does the posting. Obviously this is bad for security as if you are importing a feed that one day contained an inline script or an OBJECT that loaded a virus you could be infecting all your visitors.

2. The other safer way is to add the tags and attributes that you want to allow into the list of acceptable HTML content that the kses.php file uses when sanitising input. At the top of the kses file is an array called $allowedposttags which contains a list of HTML elements and their allowed attributes.

If you wanted to allow the playing of videos and audio through OBJECT and EMBED tags then the following section of code can just be inserted into the array.

'object' => array(
'id'=>array(),
'classid'=>array(),
'data'=>array(),
'type'=>array(),
'codebase'=>array(),
'align'=>array(),
'width'=>array(),
'height'=>array()),
'param' => array(
'name'=>array(),
'value'=>array()),
'embed' => array(
'id'=>array(),
'type'=>array(),
'width'=>array(),
'height'=>array(),
'src'=>array(),
'bgcolor'=>array(),
'wmode'=>array(),
'quality'=>array(),
'allowscriptaccess'=>array(),
'allowfullscreen'=>array(),
'allownetworking'=>array(),
'flashvars'=>array()
),

Obviously you can add whichever tags and attributes you like and this is the preferred way in my opinion of getting round this problem as you are still whitelisting content rather than allowing anything.

It took me quite a while to get to the bottom of this problem but I now have all my automated feeds running correctly importing media content into my blog. Hopefully this article will help some people out.

Creating Twitter Mashups

Using Free Online Tools to Create Twitter Mashups

If you are into Twitter then you may have wanted at some time to automate some tweets. There are many reasons you may want to do this such as creating new content quickly or providing an online presence when your not physically around. You may want to follow certain hash tags and retweet the same post with new hash tags or you may want to auto retweet relevant messages from certain friends or lists. Whatever reason you require this feature for you are not alone and there are already many tools out there that you can purchase to provide this functionality.

If however you don't want to spend any money and want to learn the necessary skills to do the job yourself then there are also many ways to accomplish this quickly. I managed to create an automated tweetbot within an hour tonight that follows a series of search terms as well as retweeting certain friends messages. It was very easy to do and I didn't have to write one line of code to get it working.

The aim of the Tweetbot was simple. I wanted content for a new Twitter account and I wanted to drive users to a site without having to manually write lots of Twitter updates. The updates I do write I wanted to retweet with new hash tags as well as retweeting certain relevant posts from friends.

By creating this Tweetbot I am delivering my earlier posts to a new audience who maybe following different hash tags as well as driving new users to my site by retweeting other related tweets automatically. I will see how this method works in terms of generating traffic over time but from a development point of view it was quite simple to set up.

1. Create a Yahoo Account if you haven't already got one.

2. Create a Twitter Account if you haven't already got one.

3. Find your source data which in my case was a mixture of Twitter RSS feeds. If you want to access a feed relevant to search terms then you can use the search.twitter.com interface. For example the following RSS is for the search term Javascript:

4. To combine the input feeds, analyse the content and reformat them into one output feed we are going to a great on-line tool called Yahoo Pipes. This free tool allows you to do lots of cool operations on various forms of input data with a drag n drop interface. You do this by creating a visual flow diagram with multiple steps going from your input source which can be one or more feeds through various filters and then to the pipe output. Each filter can be set to look for terms within the posts e.g you may want to only include posts that mention the word "javascript" or "SQL" in the title or the content. Or you may want to only include the post if it doesn't mention certain words or terms. You can do quite complex operations including using regular expressions to reformat the content, sort operations and clean-ups to remove duplicate content. The following article explains it quite well:

http://ryanpricemedia.com/2009/02/03/howto-yahoo-pipes-auto-retweet-recipe-for-twitter/

4. Once your pipe is saved you will be able to access it in RSS format from the supplied URL.

5. To get this data into your Twitter account you can either use the Twitter API and write your own code to parse the feed on a timed interval and add the posts to your twitter account e.g by using a CRON job that runs a Perl, PHP, VBScript:

http://apiwiki.twitter.com/Open+source

6. Or you could do what I did which is use one of the free Twitter feed applications that are available on the web such as twitterfeed.com. They obviously do something similar to point 5 in that you specify the RSS feed you want to import (e.g your Yahoo Pipe RSS feed) and set some configuration options such as the Twitter account details, number of items to parse, extra content to add to the posts etc.

The only downside with using twitterfeed.com is that the job only runs every half an hour which may not be a small enough interval for you. They obviously have a CRON job that runs every 20 minutes or so and parses all the feeds setup through the application so its not surprising that they cannot offer a smaller timescale however this does prevent you from over tweeting.

There are other apps out there and it would be quite simple to create your own Twitter app if you so wished with very little script, a timed job and the relevant input files.

I managed to create my PIPE within an hour and it had some quite complex content parsing and reformatting functionality so I was surprised at how easily I could do what I wanted without resorting to writing my own code. Try it yourself.