Posted:January 15, 2007

The Semantic Web Rocks!

Pandora’s Music Genome Project is Another Example of the Pragmatic Semantic Web

An initiative now gaining buzz — and one that doesn’t once mention the semantic Web — is showing how metadata characterization can bring new discovery and benefits to music listening. Pandora, which bills itself as your personal DJ, has a fantastic online music listening experience organized through a series of random shuffle-play “channels,” including those you can design and share yourself. The system presently contains more than 400,000 songs from more than 20,000 artists, most contemporary.

The Music Genome Project is the basis for the system’s recommendations. It was started by a group of musicians in 2000 with the aim of assembling the most comprehensive database of music analysis yet created. According to its Web site:

Together we set out to capture the essence of music at the most fundamental level. We ended up assembling literally hundreds of musical attributes or “genes” into a very large Music Genome. Taken together these genes capture the unique and magical musical identity of a song – everything from melody, harmony and rhythm, to instrumentation, orchestration, arrangement, lyrics, and of course the rich world of singing and vocal harmony. It’s not about what a band looks like, or what genre they supposedly belong to, or about who buys their records – it’s about what each individual song sounds like.

The basic Pandora service is shown in this screen shot:

I’ve created my own radio “channel” emphasizing Eric Clapton and specifically his blues interests with other blues artists (Clapton + Friends). The more you listen and refine, the better the shuffle play delightfully surprises. Pandora offers both free, persistent channels (with advertising) and (nominal) paid channels without advertising. (Refinement is not strictly necessary, but it overcomes the licensing limitation that prevents one from skipping too many songs in a given hour.)
Here’s the email that Pandora creates to send to friends (plus the specific reference to my Clapton channel) that you are welcomed to try out:

Hi,

I wanted to let you know about Pandora, a free internet radio site based on the Music Genome Project. It helps you discover and listen to great music. Here’s the link if you’d like to check it out:

http://www.pandora.com

Just tell Pandora the name of your favorite song or artist, and it will create a radio station that plays songs with similar musical attributes.

Here’s a link to my profile page. From this page, you can listen to my stations and check out new music I’ve found on Pandora:

http://www.pandora.com/people/mike67150

BTW, I’d like to thank a posting by Michael Daconta for turning me on to the Pandora service.

Posted:January 9, 2007

More Than 30 Useful Semantic Web Firefox Extensions

Early Progress in the Use of Firefox as a Semantic Web Platform

This AI3 blog maintains Sweet Tools, the largest listing of about 800 semantic Web and -related tools available. Most are open source. Click here to see the current listing!

The other day I posted a general status and statistical report on the growth and implications of Firefox extensions. This post presents more than 30 of those nearly 3,000 extensions that may have usefulness in areas related to the semantic Web. I welcome any additions.

These same extensions have also been added to an update of the Sweet Tools listing, which has now grown to more than 350 tools.

Please note that because the spreadsheet is hosted by Google, you must copy the URL to your address bar rather than clicking directly (direct clicking is anticipated in future versions of the Google spreadsheet; now works):

http://spreadsheets.google.com/ccc?key=0AqUZpo78do-GcEdGU1NTWk1nUU56ZVBGbjc0LTJ4TlE&hl=en

I should mention that I have seen some commentary within the semantic Web community of the desirability of compiling “best of” or “Top X” tools listings for the semantic Web. While such lists have their place, they are no substitute for comprehensive listings. First, semantic tools are still in their infancy and it is premature to bestow “best of” in most categories. Second, many practitioners, such as me, are working to extend and improve existing tools. This requires more comprehensive listings, not narrower ones. And, last, what may ultimately contribute to semantic meaning on the Internet may well extend beyond semantic Web tools, strictly defined. An ivory tower focus on purity is not the means to encourage experimentation and innovation. Many Web 2.0 initiatives, including tagging and social collaboration, may very well point to more effective nucleation points for expanding semantic Web efforts than W3C-compliant efforts.

These are some of the reasons that I have been happy to include simple Firefox extensions or relatively narrow format converters for my listings. Who knows? You never know when and where you might find a gem! (And I’m not speaking solely of Ruby!)

Posted:January 8, 2007

Firefox, Extensions and the Stealthy Revolution

A New Generation of Browser Wars is Brewing

Since its release in early 2004 (with version 0.8), Firefox has achieved phenomenal success, passing the 200 million download mark in late July 2006 and now estimated at the 282 million level or so (you can see or get a copy of the counter from Spread Firefox). Though there are seasonality factors and growth, Firefox downloads are now on the order of 3.5 million to 4 million per week.

After creaming Netscape in the browser wars of the late 1990s, it is widely acknowledged that Microsoft left Internet Explorer to languish for about five years or so, giving the opening for Firefox (and earlier Opera, though at much lower market share) to gain a toehold with fresh innovations. Some of the innovative hallmarks of early Firefox were tabbed browsing, broad operating system (OS) compatibility (Linux, Windows, Mac), constant improvements, and full and complete adherence to open standards and code access.

Though, of course, downloads by no means translate into actual users (many download and then abandon and many downloads are for version upgrades), nonetheless various independent market research firms estimate steady market share gains for Firefox. According to a report last week in ComputerWorld:

Propelled by the release of its Version 2.0 in October, the free Firefox Web browser saw almost a 50% increase in use during 2006, according to one Web measurement firm. The open-source Firefox browser was used by 14% of computers online at the end of 2006, according to Aliso Viejo, Calif.-based Net Applications. That was 46% higher than its 9.6% share of the browser market at the beginning of the year.

General consensus views are that actual Firefox market share is on the order of 12% to 15% currently.

Microsoft is fighting back, with its recent release of IE v 7 adding tabbed browsing and many of the innovative features first brought by Firefox. (This has also resulted in some hilarious send-ups, such as this mock MS site touting the purchase of Firefox). If nothing else, Firefox has helped add new competitive juice to the browser market. But, there is an even more important stealthy factor underlying these trends that bodes very well indeed for Firefox’s future and ongoing threats to Microsoft.

Stealth Extensions Growth

The Mozilla team when initiating Firefox adopted a very prescient stance: to completely open up, modularize and simplify the architecture and to publish clear and easy guidelines for extending it. This stance enabled a lean-and-mean initial browser download but more importantly provided an inviting framework for extending the system through new themes and functionality. These add-ons initially started quite slowly and first consisted of infrastructure extensions. Then, as Firefox code and documentation matured, a broader group of developers began to also see this farsighted vision and began contributing their own extensions.

Today, Firefox has just passed the 2,200 mark for extensions as maintained by its own add-on directory service. This growth is accelerating, as my figure constructed from Mozilla’s online data shows:

My own research suggests only about 90% of available extensions are listed officially on the Mozilla add-on site. The remaining are company- or Web-site-specific extensions or are experimental ones maintained (often largely) by universities. It is perhaps likely that there are about 3,000 or so extensions (separate from another 1,500 or so themes) currently extant.

What is most notable with recent trends is the growth — the number of extensions has grown by what I estimate to be 123% in the past twelve months (based on Mozilla directory data) — and the comprehensiveness and sophistication of the new offerings. Extensions are now being added at about the rate of 10 per day! and in every conceivable subject area.

As with other aspects of the Internet, extension popularity follows the typical power curve. The most popular of the extensions, such as ad blockers or video download assistants, can reach 150,000 per week or more. Quite a few extensions exceed millions in total downloads and some with many version upgrades exceed 10 to 20 million downloads. The distribution by rank popularity and downloads for these 2,200 extensions is shown in the figure below:

Again, using Mozilla data, extension downloads are on the order of 3 million per week, or nearly one extension per standard Firefox download. These extensions are growing in popularity and ubiquity and some users have documented adding 200 or more extension to their basic Firefox package. (Of course, such numbers are absurd per user, and rational means for managing and organizing multiple extensions are also emerging.) Indeed, I will shortly publish another list of about 30 extensions of specific benefit to semantic Web browsing and tasks. Extension bundles of benefit to every need and interest can easily be found.

The Role of the Browser as Platform

What is most compelling about these trends is the emerging centrality of the browser as the dominant software application in most users’ computing lives. This is part of the ongoing trend to Web OSs, as my earlier post on Parakey noted (whose founders are Firefox developers with a strong background in XUL). Firefox is truly notable for the beauty and clean design as a platform for hosting Internet applications. Using XML, XUL and its chrome files, virtually every aspect of the Firefox platform is open to extension. The Javascript examples and the fact that many of the available extensions are also fully available in source code with non-limiting open source licenses provides many examples and exemplars for still further extension innovation.

So, while Microsoft may be able to match browser-wide feature innovations such as tabbed browsing, unless it chooses as well to open the IE platform to a similar extent (granted a difficult task given the inherent proprietary architecture), I believe Microsoft will be hard pressed to maintain its dominant browser market share under assault from the global developer community. Not only are we seeing the democratization of software development through open source, we are also going to increasingly see the democratization of programming as non-programmers in the conventional sense embrace the tools and techniques being innovated by the likes of the Firefox community.

Posted:January 5, 2007

Permanent Sweet Tools Listing — 420+ Tools and Counting!

AI3’s Comprehensive Listing of Semantic Web and Related Tools

This AI3 blog maintains Sweet Tools, the largest listing of about 800 semantic Web and -related tools available. Most are open source. Click here to see the current listing!

Since my first posting of 175 semantic Web tools and then an update to 250, the listing has become quite popular and an apparent asset to the semantic Web community. While this AI3 tools listing is not as precise and restricted as the “official” ESW one on the W3C’s Web site, it does contain useful adjunct tools in such areas as parsers, natural language processing, wrappers and the like that are also of potential usefulness to semantic Web practitioners.

Because of the popularity of this listing, I decided to make it easier to access and update by others in the community. Thus, I converted the listing to a permanent feature of this blog (see the Sweet Tools link to the upper left in the Main Links area) as well as posted a publicly accessible Google spreadsheet link (requires Google account!) for direct updates.

Current Listing

As of the date of this posting, I have added 42 new tools since version 5. The listings are posted as an Exhibit-based lightweight structured data publication (as explained here), which allows filtering, sorting and current statistics.

I continue to characterize the listings by: 1) FOSS (free and open source software), with about 90% of the listings being so; and 2) a categorization of the tool type. Currently, there are 27 categories listed, of which some of the tools are surely mis-characterized. If you add a tool (see below), please try to use these categories or suggest a new one to me directly.

I should also note that I track about 250 companies that provide semantic Web software (generally) under license fees. Most of those companies are NOT included in this listing; I may add these at a later point, but such tools are generally quite expensive. (To learn more about these companies, you may want to try SweetSearch, and then restrict by the ‘Company’ facet.)

Finally, you might be interested in the open source popularity of these listings. Raphael Volz published a popularity analysis of the earlier 250 tools listing based on SourceForge statistics; very interesting reading! Thanks, Raphael.

Selective v. Comprehensive Listings

Two Ways to Contribute

If you have new tools to add, corrections to current listings, or any other suggestions, you have two ways to contribute. The easiest way is to post a comment to this entry and I will update the listing based on your input. The second way is to access the Google spreadsheet link itself and make changes directly. I will continue to keep this spreadsheet public unless spam proves to be a problem.

Thanks for your interest and Enjoy!