One of the things I noticed was the large amount of duplication in functionality that my site now has due to using these 3rd party add-ons and tools. There is no doubt that these widgets offer some great functionality with little or no cost and development time and they have enhanced the sites usability. However using Googles AJAX API, jQuery, add-this and snapshots and my own core library strictly.js I now have 5 separate JS files being loaded that handle events, load scripts dynamically and other various DOM manipulation functions.
I don't know what the answer is as its too much to ask 3rd party developers to all develop their add-ons with one framework or to ask them to create multiple versions of their widget for each framework. Plus its very likely that any developer who creates a nice add-on will want to write their own code anyway as any coder worth their pay thinks they know best and in most cases they choose to encapsulate any helper functions into the add-ons core files.
Now you could spend a lot of time creating local copies of these add-ons and rewriting all the code to share as much as possible but this would be futile and take a lot of effort. Plus every time a new version came out you would have to go through the same process or otherwise stay with an old version of the software.
This started me thinking around the possibility of some sort of CDN which any developer could access and upload to that would hold files that met the requirements of a standardised API. Developers of new add-ons instead of recreating the wheel could build their widgets using these central objects. It would then be up to the site owner to decide which version of the object he wished to use but as long as any add-ons used on their site implemented this standard API the code would all work.
For example thinking about the common need for event handling by all widgets. Say I am developing a new widget that shows previews of other sites (like the Snapshot add-on). Instead of writing new code to handle event management (add, remove, onload, DOM ready etc) I implement this standard API that has a specific naming convention. There can be multiple objects written to handle events on this CDN and they all differ in their internal workings but they all support the same interface and therefore it would be very easy for a developer to hook into them or change the core framework. An example would be that for any event object written to support this API to add an event they would follow this convention: element.addEvent(type,function,capture).
The main framework developers would all have to standardise their naming conventions if they wanted to use the API but it would mean any site owner using multiple add-ons could choose one core framework for their sites engine and as long as all the add-ons and widgets were built to use the standard API it wouldn't matter if they changed the core framework from JQuery to Prototype the site would continue to work and without the need for code duplication. 3rd party widget developers could specify in their documentation which framework they felt the widget worked best with but it wouldn't depend on that framework and a site owner would make their choice without worrying about breaking the add-ons.
It just an idea that I have just had and it would obviously need lots more thought and the support of the major framework developers as well as any 3rd party widget developers. However until the time comes when some sort of standardised central codebase is implemented and widely used it looks like code duplication and large files are here to stay. Therefore to mitigate this issue site owners will need to make sure all these files are optimised as much as possible through compression, minification, local hosting or CDNs, caching and other tweaks.
This article is a great resource for covering this in detail: http://websitetips.com/optimization/