Mining for minerals and oil has always been a risky activity, but the rewards are really worth it if you are lucky, careful, and work quickly. Canada’s economy is highly dependent on resource extraction. My grandfather (on my father’s side) died in a mining accident when my father was only 9 – this was at the height of the depression when the price of silver had dropped so low that he had to lay off all his workers and was doing the work by himself. He was not one of the lucky ones. There is a law in Ontario that allows anybody to stake a claim on your property if they want to extract minerals from it. I believe they must reimburse you for certain expenses related to loss of land use, especially if they put in a road to get to the claim, but basically as a landowner you have very few rights. Too bad for the land owners, I suppose, but those minerals must be had. The profit motive clearly drives this industry – profits for the miners, taxes for the government – and it’s easy to see why. The money is real, it’s right there, go get it.
Mining for information is established on a much less solid foundation because the nature of the information you will uncover is by definition unknown, and hence the potential profits are also unknown. I’m not talking about business information systems that analyze sales trends and help identify emerging trends so you can make money by recognizing and keeping up with the trends. These are clearly tied to the profit motive and the business imperative is undeniable. I’m talking about the other types of information in the system such as project statistics, emails, twitter feeds, source code repositories, feature requests, performance statistics, operations logs, and so on. Twitter feeds? Didn’t I state in my previous log that I didn’t see much use for them? Yes, I did. But I also said there is some potential use for twitter, which is to recognize social circles. I could also add that tracking of communications patterns may yield some useful information. For example, you might be able to track morale in the company based on traffic patterns. This is not crazy. Private investigators, spies, and police investigators have long relied on phone call traffic patterns as a useful information resource. Unfortunately, companies may choose to use traffic patterns for more nefarious means – for example, to quash union organizing efforts, to be warned when an employee seems poised to quit, or to accumulate “the file” on bad employees. That’s reality in an information society, folks. Get off the train if you don’t like where it’s going, or even better, lay tracks towards a better destination.
There are plenty of good things you can glean from communication traffic patterns. Throw in analysis of the actual content (not just who contacted whom and when) and you could see trends as they emerge instead of reacting to them well after the fact. Some day in the not too distant future, researchers will start understanding exactly how the information will affect your bottom line. Actually, that’s a bit of a mis-statement. Researchers have a long history of tying their recommendations to statements about profitability – also known as the infamous return on investment. Unfortunately people get out the silver bullets and crucifixes when you mention return on investment, or ROI, because sales people are often a little too generous on the return portion and not sufficiently attentive to the investment bit. And to be fair it is often difficult to attribute positive events in the balance sheet to initiatives started many years earlier. For example, Cummins Inc. is a US-based manufacturer of engines and generators. Earlier this decade they embarked on a software product line initiative that has yielded concrete benefits in terms of engineer productivity and product agility, and the stock price has doubled over the past year. But as of last year when I attended the software product line conference in Ireland, they couldn’t prove that their profits were related to the SPL initiative, and perhaps they never can.
I believe that efforts to prove the benefits of information mining activities will start yielding believable results once we wrap our arms around all the data, not just specific snapshots of the data. Unfortunately, this is a time-intensive and often unrewarding activity. It’s a bit of a catch-22 situation: You can’t prove you need all the data until you have all the data. The best hope is to deploy automated agents that will collect the data for you since it can’t be done by hand. Agents shouldn’t cost much to maintain. Only problem is, we don’t really have anything that works yet and it may be a while before we do. There are plenty of initiatives in progress, which I hinted at in my previous post but which I haven’t yet compiled enough information about. So stay tuned – I’ll get to it eventually.