Posted:December 14, 2005

A Walk in the Snow

You are what you eat … You know what you experience … You know what you know … But you don’t know what you don’t consider ….

I just walked outside my home and spent a glorious 30 min this evening viewing the snow, the sky, the vistas, the lights, all of it.

I’m about ready to head with my family to many days of skiing in Colorado. We have done so before in many places and many states. In every instance, I can remember the physical moments and the visual moments. Some of those visual moments were quite striking: looking over the back side of Big Mountain into Glacier, overlooking Lake Tahoe, looking north into the broad stretches of Canada.

But what truly blew my mind tonight was that the snowy ground and the vistas from my own backyard were as beautiful as what I have seen elsewhere. Granted, I’ve got open space from my back porch here in Iowa City, but, face it, most anyone who does not live here would not believe there are "world class" views across these prairies. My own backyard view combines pastoral scenes with some suburbia and with views no higher than one or two hundred of feet in topographic change.

But it is beautiful … and it is peaceful … and it is striking in its impact.

So, I think the following: I’m about ready to head to a known wonderful place (in the Rockies) with known vistas and known aesthetics. In that regard, I am a fortunate person and am willing to spend many dollars to do so. But, then, I also think: My own backyard is pretty cool. Actually, more than cool: Beautiful.

My family and I will have a great time and we will enjoy the skiing as we always do. But the magic of our own vistas of snow this night, the lights at night and the dark rolling hills and naked branches are pretty cool here as well.

The short of it is this: You don’t miss your water ’til your well runs dry, and you sometimes don’t even know what your true source of water is.

For me, tonight, it is my backyard. I hope, if you care to look around and find the right viewpoint, that maybe it is the same for you ….

Drink deeply this Christmas ….

The Single Database Chimera

I just finished participating in a discussion that has mirrored many others I have observed in the past: We have a complicated problem with much data before us, and we don’t know where it may evolve or trend. Can we architect a single database schema up front that handles all possible options?

Every programmer or database administrator (DBA) will recommend keeping designs to a single database, schema and vendor. It makes life easier for them.

However, every real world application and community points to the natural outcomes of multiple schemas and databases. This reality, in fact, has what has led to the whole topic of data federation and the various needs to resolve physical, semantic, syntactic and other schema heterogeneities.

Designers can certainly be clever with respect to anticipating growth, changes as seen in the past and so forth. Leaving "open slots" or "generic fields" in schemas are often posited and perhaps may allow for a little bit of growth. Also, perhaps quite a bit of mitigation for schema evolution can be anticipated up front.

But the reality of diversity remains. The semantic Web and proliferation of user-generated metadata will only exacerbate these challenges. Simply talk to the biological or physics communities of what they have seen in finding a single "grand schema." They haven’t been able to, can’t, and it is a chimera.

Thus, smart design does not begin with a naive single database premise. It recognizes that information exists in many forms in many places and in many transmutations from many viewpoints. And what is important today will surely change tomorrow. Explicit recognition of these realities is critical to successful upfront information management design.

Viva la multiple databases!

Posted:December 13, 2005

Another Good Semantic Web Series

I just came across an easily readable and accessible short series on the semantic Web by Sunil Goyal of the enventure blog. The four-part series consists of:

Part 1 — introduction and overview of various Web services
Part 2 — the challenges of data integration
Part 3 — RDF and OWL data models and service-oriented middleware, and
Part 4 — user applications, enterprise systems, research applications and themes and services.

If you are new to this topic, you may find this series an easy first introduction.

Posted:December 5, 2005

GSA Open Source Initiative

Earlier posts have noted the near-term importance of the federal government to integrated document management and integration of open source software. A recent article by Darryl K. Taft of eWeek.com titled "GSA Modernizes With Open-Source Stack" indicates the lead role the General Services Administration will play, at least on the civilian side of the government. According to the article:

George Thomas, a chief architect at the General Services Administration, said the GSA is leading the effort to deliver an OSERA (Open Source eGov Reference Architecture) that will feature foundational technologies such as MDA (Model Driven Architecture), an ESB (enterprise service bus), an SOA (service-oriented architecture) and the Semantic Web, among other things.

OSERA deserves close tracking as the federal government implements these standards. GSA has set up a Web site on OSERA that is still awaiting content.

Posted:December 2, 2005

Large-scale Intelligence Analysis

A recent article by Cheryl Gerber in the November 21 issue of Military Information Technology online on "Smart Searching" provides a useful overview of issues and leading vendors dealing with large-scale issues of content search and discovery. Some of the useful vendors covered in this article include: Endeca Technologies, Basis Technology, Inxight Software, Insightful, Attensity, Convera, SRA International (NetOwl), ClearForest and BrightPlanet.

The focus of these efforts in the Defense Intelligence Agency is characterized by Gerber as:

The unique requirements of defense intelligence analysts are refining search technology down from mass production, with its vast and sometimes trivial outcomes, to more guided, dynamic navigation able to produce results that are both inclusive and relevant.

As one of the largest collectors of information on the planet, the Defense Intelligence Agency (DIA) is responsible for amassing and analyzing all sources of human intelligence in the field from all information types in a multitude of languages.

"This forces us to deal with huge volumes of data. It's an enormous challenge," said a senior DIA official.

The task is indeed a massive one. Sources of intelligence in the field include feeds from UAVs, intelligence, surveillance and reconnaissance data from a vast array of sensors and overhead platforms, signal intelligence, satellites, film and video, not to mention all the data from the open source world. "We need to manage all that data and make it available as quickly as possible to analysts," the DIA official said.

The intel community, as with others forming in the commercial sector, is also relying on community standards for metadata transfer and management. In the case of the DIA, these standards are being provided by the Intelligence Community Metadata Working Group (ICMWG), which is charged with establishing standards for the tagging of all data used by DIA systems.

In the article, BrightPlanet’s Duncan Witte commented on the importance of having the abilities to "organize, manage and distribute the huge volume of information as well. You need various specialties that allow collaboration with teammates and effective distribution of information."

This article again affirms that the federal intelligence community continues to assume the lead in large-scale content discovery and evaluation.

As the article notes, the DIA maintains a steady push toward technology improvement. "We try to do the best we can with the volumes. In-house we have a lot of expertise on search algorithms and text analysis. But we need to do a better job of combing through the massive volumes of information to find that which is interesting and nontrivial in a way that leads to knowledge discovery. We need better information retrieval through machine understanding of the semantic meaning of text, regardless of language," the DIA official said.

Main Links

Search

Month: December 2005