John Newton (co-founder formerly of Documentum, now of Alfresco) puts a telling marker on the table in his recent post on the Commoditization of ECM. Though noting the term "enterprise content management" did not even exist prior to 1998, he goes on to observe that expansion of the definition of what was appropriate in ECM and the consolidation of the leading players occurred rapidly. He concludes that this process has commoditized the market, with competitive differentiation now based on market size rather than functionality. The platforms from the leading IBM, Microsoft and EMC-Documentum vendors all can manage documents, Web content, images, forms and records via basic library services, metadata management, search and retrieval, workflow, portal integration, and development kits.

If such consolidation and standardization of functionality were Newton’s only point one could say, “ho, hum,” such has been true in all major enterprise software markets.

But, in my reading, he goes on to make two more important and fundamental points, both of which existing enterprise software vendors ignore at their peril.

Poor Foundations and Poor Performance

Newton notes that ECM applications are never bought based on the nature of their repositories, but an inefficient repository can result in the rejection of the system. He also acknowledges that ECM installations are costly to set up and maintain, difficult to use, poorly performing and lack essential automation (such as classification). (Kind of sounds like most enterprise software initiatives, doesn’t it?)

Indeed, I have repeatedly documented these gaps for virtually all large-scale document-centric or federated applications. The root cause — besides rampant poor interface designs — has been in my opinion poorly suited data management foundations. Relational or IR-based systems both perform poorly for different reasons in managing semi-structured data. This problem will not be solved by open source per se (see below), though there are some interesting options emerging from open source that may point the way to new alternatives, as well as incipient designs from BrightPlanet and others.

The Proprietary Killers of Open Standards and Open Source

Service-oriented architectures (SOA), the various Web services standards (WS**), the certain JSRs (170 and 283 in documents, but also 168 and others), plus all of the various XML and semantic derivatives are moving rapidly with the very real prospect of “pluggability” and the substitution of various packages, components and applications across the entire enterprise stack.

In quoting Newton’s case at Alfresco, by aggregating these existing open source components they were able to get their ECM product ready in less than one year:

• Spring – A framework that provides the wiring of the repository and the tools to extend capabilities without rebuilding the repository (Aspect-Oriented Programming)
• Hibernate – An object-relational mapping tool that stores content metadata in database and handles all the idiosyncrasies of each SQL dialect
• Lucene – An internet-scale full-text and general purpose information retrieval engine that supports federated search, taxonomic, XML and full-text search
• EHCache – Distributed intelligent caching of content and metadata in a loosely coupled environment
• jBPM – A full featured enterprise production workflow and business process engine that includes BPEL4WS support
• Chiba – A complete Xforms interface that can be used for the configuration and management of the repository
• Open Office – Provides a server-based and Linux-compatible transformation of MS Office based content
• ImageMagic – Supports transformation and watermarking of images.

Moreover, the combination of these components led to an inherent architecture including pluggable modules, rules and templating engines, workflow and business process management, security, and other enterprise-level capabilities. In prior times, I estimate no proprietary-based vendor could have accomplished this for ten times or more the effort.

Similar Trends and Challenges in the Entire Enterprise Space

Newton is obviously well placed to comment on these trends within ECM. But similar trends can be seen in every major enterprise software space. For virtually every component one can imagine, there is a very capable open source offering. Many of the newer open source ventures are indeed centered around aggregating and integrating various open source components followed by either dual-source licensing or support services as the basis of their business models. At its most extreme, this trend has expanded to the whole process of enterprise application integration (EAI) itself through offerings such as LogicBlaze FUSE with its SOA-oriented standards and open source components. Initiatives such as SCA (service component architecture) will continue to fuel this trend.

So, enterprise software vendors, listen to your wake up call. It is as if gold dubloons, pearls and jewels are laying all of the floor. If you and your developers don’t take the time to bend over and pick them up, someone else will. As Joel Mokyr has compellingly researched, the innovation of systems or how to integrate pieces can be every bit as important as the ‘Aha!’ discovery. Open source is now giving a whole new breed of bakers new ingredients for baking the cake.