Scott Thurm recently wrote a piece in the WSJ Marketplace section – Now, It’s Business By Data, but Numbers Still Can’t Tell Future.
He talked about the growing trend of trying to run companies more analytically, more “by the numbers”, and the success some of those companies have had with this approach including those profiled in Tom Davenport’s book “Competing on Analytics” (reviewed here). He quoted Robert Sutton (one of the authors of Hard Facts, Dangerous Half-Truths And Total Nonsense, reviewed here) who contrasted running a business by the numbers with running it based on “faith, fear, superstition and mindless imitation”! However, Scott then goes on to identify two key risks inherent in an analytic or data-driven approach.
* Change upsets the basis for the analysis
* Too much focus on analysis stifles the creativity needed for innovation and tomorrow’s growth
These are both good points and made me think about enterprise decision management(EDM) in this context. Now Tom Davenport talked about the need to make “quick, accurate decisions on an industrialized scale” when he reviewed Smart (Enough) Systems. These kinds of high-volume, operational decisions are the focus for EDM and change to the environment in which you are making those decisions must be considered. No decision can be automated in a way that will ensure it remains effective indefinitely – changes to competitors, markets, products and economic conditions will conspire to ensure it degrades over time. If you are lucky, it might degrade gently. If you are unlucky, some sudden shift could ruin you.
This need to manage and improve decisions in the face of change is why challengers is so important to the successful adoption of decision automation. With adaptive control you constantly test your current decision making approach against challengers to see if any of them work better. This helps both find better approaches and spot when your existing approach is no longer optimal. An infrastructure for adaptive control also allows you to move into true experimental design where you are systematically checking a large number of potential approaches and aiming for continuous optimization.
Some changes are too sudden for this approach, however, as the results of many decisions take a finite length of time to collect. A very rapid change might mean you are in trouble before the results show it. Using decision simulation techniques, and a robust model of what influences the decision, many organizations are now running scenarios (such as much higher interest rates or a bad hurricane season) to come up with the rules and analytic models that work best in those circumstances. These decision approaches can be kept on the shelf ready to go in case one of those major changes should occur. Even this does not completely solve the need to respond to unexpected change, for which a general focus on agile information systems is about all you can do, but it limits the circumstances in which you will have no response beyond “gut feel”. Now that I have finished Harry Potter 7 (of which more later), I am also in the middle of reading Nassim Taleb’s The Black Swan: The Impact of the Highly Improbable and I will write up a review and some thoughts on managing “black swan” events sometime soon.
Even with adaptive control you run the risk that you are perfecting today’s business rather than thinking creatively about tomorrow’s possibilities. However, when you are dealing with massive scale (millions of accounts, hundreds of thousands of customers etc), you really have to be able to model the impact of your creativity before you put it into production. Changing a web page layout may be easy enough to try and see if it improves people’s user interaction, but introducing new pricing or a new product without having some idea what impact if might have on other products, shelf space, marketing campaigns etc is probably foolish. I would also suggest that creativity and analytics can go hand in hand, as I have discussed before in my review of Malcolm Gladwell’s Blink and of Larry Rosenberger’s “Future of Analytics” presentation at InterACT last year.