Before you yawn and turn the page, ask yourself whether you are completely confident that the business rationale for your projects is entirely justified, and that there is clear understanding between you and your customer on the economics of your IS investment? The truth is that the ROI for most IS application delivery projects, irrespective of whether they are package implementations, application development, or application integration, are based on little more than blind faith!

At a recent CBDi Forum Special Interest Group (SIG) meeting on exactly this topic, there was general agreement that most projects are justified by presenting the facts in a manner most likely to win approval, rather than on a sound, quantifiable and tangible set of economic arguments. It seems that the general state of ISs delivery practice is still sufficiently immature that justification and measurement systems, common throughout industry and commerce, remain unused by IS projects.

The result of this sorry state of affairs is that software projects remain as uncertain as ever. The failure rate of software projects, measured as those projects that fail to deliver to within 20 per cent of the agreed time and quality objectives, remains extremely high. As a result, the business community has lost much confidence in IT or IS departments, which has resulted in two clearly observable trends – indiscriminate outsourcing and short-term thinking. Widespread lack of confidence in the IT departments’ ability to deliver what the business needs by means of custom solutions has led, over the past few years, to massive replacement of core systems with packages. Yet, the reality of package implementations for many has been heavy time and cost over-runs, and deep dissatisfaction with the resulting business fit and ongoing adaptability. Activity that has remained in-house has often been run in a highly tactical manner that has focused on lowest cost and fastest delivery timescale.

It seems that the IT community, programme managers and software engineers have consistently failed to convince their customers that quality is a variable. Always taking the approach of spending the lowest amount for fastest delivery is ridiculous. Most people would quickly reject this thinking when making personal purchases such as hi-fi’s, kitchens, homes, and automobiles. Does everybody always buy the cheapest kitchen units they can find? Managers in other disciplines will take relative quality into account when purchasing plant, equipment, and services, making sensible judgements that balance functionality and Total Cost of Ownership (TCO). Yet, when it comes to IS departments, fastest delivery and lowest cost seems to be the pervasive management thinking.

At one of my tutorial workshops recently, I was asked the question "will it be possible to manage the component-based application environment more effectively than the conventional systems environment? Can we expect increasing sophistication which parallels evolution in say production engineering?" My answer was that yes, the componentised environment does lend itself to better management, primarily because there is, by definition, more formal separation of, for example, units of manufacture, assembly, and replacement.

That this, in my opinion, is already leading to the separation of manufacture and assembly, supply and consumption, and the very separation of these processes encourages discipline and, in particular, formal economics and metrics that are normal commercial practice where buying and selling take place. Sellers want to protect their interests and buyers want to ensure that they are getting value for money. This automatically leads to formality and extensive consideration of fitness for purpose.

However, I realise that, in retrospect, my argument is somewhat flawed. The assessment of ‘fitness for purpose’ for many of the large number of Enterprise Resource Planning (ERP) purchases and other application packages acquired over the past few years, has clearly been inadequate in, at least, a significant proportion of these cases. If buyers ended up with a purchase that was not optimally suited for purpose, then the procurement and planning processes must be at fault.

There may be many issues that we can identify as playing a part in this problem. Software is enormously complex and conceptual and difficult to comprehend. Businesses are also enormously complex and more problematic, often changing constantly. Finding the authoritative personnel who can say what the requirements of the business are is not always easy. Senior management has a broad strategy in mind, but generally cannot articulate the concomitant operational details. Operational management knows what happens today, but probably will not be fully aware of potential for change. Business management often does not trust IT management to deliver what is required, and is likely to favour outsourcing if it has recently had a bad experience.
The converse is also true.

It seems that one answer to this extremely difficult problem, as I observed to my workshop tutorial delegates, is to have greater focus on the economics and metrics in these situations. Instead of management by blind faith, we need some management disciplines that allow us to have confidence that we are making reasonable decisions. Where components assist, is that the problem is potentially made more manageable by being comprised of separable units of management and delivery. These can, of course, be procured from different sources, adhere to different levels of quality, and provide a basis for measurement and management understanding, which is somewhat finer grained than say an entire ERP application package, yet comprehensible in business terms because they offer a number of business services.

Applications are fundamentally assets. An asset is something that provides benefit to the business and is planned, procured, implemented, utilised, and eventually retired and replaced.

There seems to be no good reason why we should not treat an information service in the same manner as any other asset. That is we should be able to establish a business case that is founded on tangible business benefit. If we have a formal statement of economic need, then we have a basis for establishing how we should procure the asset. One of the most important aspects of business justification for systems that is constantly ignored or dismissed is the lifetime cost of ownership. We are all aware that the system which is delivered on the cheap, is most likely to be the legacy application that eventually ends up costing the most in long term maintenance and replacement costs.

In Table 1 below, we suggest some key questions that need to be answered, which then drive business-oriented decisions on the asset.

Unfortunately, the experience of many organisations with information systems delivery has been so poor that economic arguments have been devalued. If the average project over-runs by 50 per cent, do you plan for that cost escalation up front, or do you allow hope to triumph over experience once again.

Of course, business managers who have overruled IS departments and bought packages, only to find their implementation time and cost estimates have also been blown sky high, will now be on the same level playing field with the IS folk.

The acquisition process, therefore, needs to be focused such that the sourcing and integration challenges are well understood, and that the estimates for build, assembly, and integration are founded on an experience knowledge-base that provides some confidence. Otherwise, the key management decisions on IS applications procurement will continue to be predicated on the basis of personal experience and judgement. In our constantly changing IT world, personal experience is potentially an unreliable basis for future decisions.

In Table 2 below, we suggest some key questions that need to be quantified, which can directly drive better procurement business decisions.

Increasingly, components are becoming available that will make the above business decisions easier, by providing more choice and more flexibility. These raise new questions of: "what are the costs of acquisition?" And: "how do you select the components that provide appropriate functionality?"

A perennial question, that organisations have grappled with, is just how to select components or applications that will provide appropriate functionality and support. The issue is that components are, in many respects, like packaged applications, they vary enormously in the degree of customisation that can be achieved.
This applies equally to components that are acquired as white and black boxes. Acquiring a black box component, other than highly generic functionality, will generally require that some customising functionality is added surrounding the black box, as shown in Figure 1 below. This might vary from simple reformatting of message formats, to extensive glue logic implementing differences in business semantics.

The key question to understand is just how the component or application is designed to support change, both at implementation time and on an ongoing basis. The component supplier will have implemented some strategy covering a number of criteria, as shown in Figure 2 below, covering the relative levels of malleability, layering, generalisation and dynamic access.

Suppliers of conventional ERP and other packages generally provide base functionality, which is designed to be customised in relatively constrained ways.

Increasingly, suppliers of components will provide both design time and run-time specialisation capabilities that will allow varying levels of customisation on these various axes. It seems entirely reasonable that suppliers should be able to provide metrics on the average installation and customising effort of individual components that demonstrate the effectiveness of the customisation capability.

It is possible that in-house application construction projects are subject to less formal metrics than all other forms of systems delivery. Very few organisations will admit to using metrics, other than the basic project management numbers of man resources per task. Estimating approaches are in common use, which use archaic techniques that are mutually exclusive with modern iterative life cycles. Component-based development, in contrast, offers significant additional management control. Users report that just the clear separation of responsibility and work allocation represented by components, limits scope creep and focuses accountability. More significantly, the inherent precision of component scope provides the basis for better estimating and management information.
For example, dependencies with other components represented by implemented services can be an important metric of the complexity and interdependency of a project.

In the past, measurement systems have concentrated on design, build, and construction projects.
With component-based development, there is a radical new model that combines build and buy (reuse) in the same project.

Even specialist supplier and consumer organisations will undertake both functions to some extent. In the component-based environment, there is considerable opportunity and demand for better management information because of the inherent structure that the component architecture presents. The opportunity to acquire at a finer grain than the package, means that there is an incentive to better understand the trade-offs between build or buy. If an acquired component is available, for say $5000, then it may make sense to acquire it, however, the cost of acquisition and integration may have a dramatic impact on the raw purchase cost. The only way to find out is to monitor the actual costs and then build these into a knowledge-base of estimating data.

The opportunity for IT organisations that can accrue from better management information is very significant. The IT organisation that can demonstrate confidence in its estimates is on firm ground when guiding the business in making application related business decisions. In the past, measurement was the preserve of the project manager, and she/he was primarily interested in resource and work related data. Today, we recommend that the IT organisation needs to take a broader view of
its responsibilities, by examining the information needs of the various roles and stakeholders in the IT organisation. In Table 3, we show a sample set of management information needs that might be viewed as critical for the architect, the (component) inventory manager, the development manager, the project manager, the analyst, and the implementor. We do not believe this list is definitive, rather it is a strawman, designed to spark interest in the topic.

The CBDi Forum has recently commenced a strand of activity to investigate metrics, measurement, and evaluation approaches in the component-based environment. The ‘Evaluation and Metrics’ SIG is planning to develop these ideas, and also to run a program to establish some empirical data that will answer some of the key generic questions posed in this paper. If you are interested in this line of investigation, take a look at the SIG area on the CBDi Forum Web site, and contact me, David Sprott: