Information evolution

The focus of my blog is information architecture for corporations, organizations and the internet. While much of the information is generated or acquired once and never modified afterwards, a significant portion of the data is used, modified and reused. Many times, even! Or at least, that’s what you hope.

Information reuse has the potential for significant productivity improvement, but it comes at a price: You must keep track of its versions. And not just the version of the thing you are looking at – you must keep track of the versions of the entire ensemble of information that you are looking at. Why? Because you’re always working on the next version(s) of the product or service, while all your customers (or consumers) are always working on a previous version. Because sometimes you have to parallelize your efforts – in effect have two or more teams working independently off the same base level. Because sometimes you need to figure out why things turned out the way they did – turn the clock backwards, as it were, and then run it forwards again to watch the evolution.

Software people have done this for decades – it’s called Software Configuration Management, or SCM for short. They knew they had to keep careful track of versions because if you didn’t then your code wouldn’t compile, or worse, it would compile but exhibit strange bugs, or worse, it would compile, test out fine, but customers would find strange bugs. Each bug a customer finds costs as much as 50 or even 100 bugs that you discover in-house.

More recently we’ve realized that other non-code artifacts, especially requirements and test case descriptions, are just as valuable to reuse as source code. Some vendors – my employer MKS being one, but others are starting to spring up – version the individual requirements, not the documents. We recognize that a given requirement has its own provenance, its own distinct lineage, and can be reused in more than one document, and therefore deserves to be versioned separately just like source code modules. But keeping track of requirements versions is not the same as keeping track of code versions. First of all, you don’t have the safety mechanism of code compilation. It’s difficult to tell that a given set of requirements is self-consistent. The simple addition of the word “not” to a requirement can mess things up pretty badly. Second, requirements come with significantly more baggage than source code – metadata, relationships to other requirements and other artifacts and location inside a document. This baggage is the reason why in my previous post I drew source code artifacts as circles and non-code artifacts as stars. Requirements are spikey. They are difficult to move around. They are beautiful and shiny, but stubborn and time-consuming. Kind of like movie stars.

But do they evolve differently? And what forms of evolution are we talking about?

It’s not Darwinian evolution, for sure. If it were evolution in the Darwinian sense then we would start with a sample population, allow it to cross-fertilize by some means, allow for occasional mutations, and let the most fertile information thrive over the generations by spawning the most, er what – children? Yeah, that’s not working for me.

In the next few posts I will lay out the basics of versioning and branching, both at an individual level and at the group or ensemble level, showing the differences between best practices and reality, and showing how source code is different from non-code artifacts. And believe it or not, we haven’t seen the last of Mr. Darwin.