Introduction

Since publishing a report¹ on the Service Oriented Process earlier this year CBDI have been busy developing the CBDI Knowledgebase in some depth in both the Service Oriented Architecture and Design and Service Provisioning disciplines of the Service Architecture & Engineering (SAETM) SO Process.

In parallel TIBCO have been refining their Total Architecture² solution development methodology to incorporate the identification, creation, and usage of services driven by business needs and business process design.

This article presents a common process framework that:

• Fleshes out the solution delivery process in a fashion that “mirrors” the service delivery process, including the concepts of solution architecture and solution provisioning
• Extends the concept of an architectural design that encompasses all three service tracks: consume, provide, and enable
• Incorporates solution deployment and operation (as well as service deployment and operation) within the enable process.

The CBDI – TIBCO Collaboration and Statement of Intent
There has been much marketing hype around BPM and SOA with very little substance. The CBDI - TIBCO process framework provides much needed industry guidance on harnessing business process improvement and solution delivery with SOA and service provisioning. The framework provides clear interfaces between the various disciplines of service orientation that promote a deliverable driven – as opposed to a task driven – process. This is very important for consistency of work and for the key service oriented principle of measurability through specification: “you can’t control what you can’t manage, and you can’t manage what you can’t specify.”³

The aim of the CBDI and TIBCO collaboration is to jointly publish a SO Process Framework that

• Identifies each of the disciplines (a significant competency which has the ability to perform one or more activities) required to deliver and manage services and solutions
• Enables interoperability by identifying the agreed deliverables that are exchanged between disciplines
• Is an open published framework that can be adopted by other parties.

It is not the intention of CBDI and TIBCO that they or other participants should jointly agree on the detailed activities, techniques and best practice guidance within any given discipline (though participants are free to collaborate on these should they wish). Rather, each discipline provides a boundary around a closely related set of activities. It is these boundaries that shape the overall SO Process, defining shared deliverables as interoperability mechanisms between disciplines, instead of defining shared implementations of disciplines.

The value to user organizations is that as a consequence they should be able to:

• Select best practice from either CBDI and TIBCO, or other participants at the discipline level
• Better manage outsourcing or distributed development though a sharper focus on the disciplines and deliverables required, without unnecessarily constraining the participants to follow a specific way of working
• Improve the interworking and collaboration between SOA efforts and software development projects
• Provide a much more practical context for their SOA efforts that is related to the business needs that drive software development projects
• Make software development projects more responsive and less costly through practical service reuse.

Process Overview

Figure 1 shows the enhanced SO Process Framework in “big picture” form. We have introduced two new disciplines and effectively factored out and extended two further disciplines. Let’s quickly review the key enhancements before moving on to discuss the improvements in some detail. We will then conclude by returning to the “big picture” and filling in the details.

Figure 1: The Enhanced SO Process Framework: Simplified View

Improvements to the Consume Track

The consume track is significantly enhanced, to incorporate full solution delivery, with two added disciplines:

• Solution Architecture and Design: To design a solution at a high level in such a way that its business process architecture and its system architecture integrate as smoothly as possible, and to identify the service requirements to enable the solution.
• Solution Provisioning: To plan the sourcing and usage of a solution (or part of a solution), to specify and design the solution and to ensure the solution (as an integrated whole) is fit for purpose. This is analogous to the concept of service provisioning.

These changes allow the full spectrum of solutions, from one which actually employs no services at all to one founded upon existing services, to be covered. Let’s briefly examine how these cycles pan out.

Traditional Solution Development

In the Total Architecture approach, solution development progresses from assessment of business improvement requirements to analysis of associated business process(es), through solution architecture and subsequent design involving a set of software components. This is a highly iterative approach in which the business process architecture and its system architecture are evolved together as seamlessly as possible. The components are assembled into a solution, which is partially certified⁴ , before being run on the platform of choice. Measurements from the executing solution are fed back to solution provisioning so that the solution’s actual achievement of the desired business goals can be fully certified. Variances from the expected results may result in solution refinement or new inputs to the business improvement activity. In traditional development the software components do not actually offer any services, and no service requirements are identified.

Service-Based Solution Development

Ideally, the development of solutions in an SOA environment would consist largely of assembling existing services into working solutions. Such an approach makes heavy use of existing services and raises requirements for new services. The solution lifecycle, as before, progresses from assessment of business improvement requirements to analysis of associated business process(es), through solution architecture and subsequent design based upon a set of software components. However in the SOA scenario there is a much greater emphasis on identifying the service requirements to enable the solution. This entails significant iteration with the provide track.

Service requirements identified within Solution Architecture and Design must be assessed by Service Oriented Architecture and Design (SOAD) at the relatively high level of service descriptions, before Service Provisioning can commence to work on any required service specifications; see box 1 for explanation of the nature of the interactions with SOAD.

Subsequently, Solution Provisioning receives certified service specifications (and possibly SLAs) from Service Provisioning. The service specifications are used as the basis for incorporating the service within the solution design, with the use of the service typically being coordinated with some form of business process orchestration.

Following Service Implementation and subsequent Service Deployment the services are published via some form of catalog. At this point the solution implementation and service assembly can proceed using the published and deployed services along with any non-service components comprising the solution. The fully assembled solution is tested and partially certified before being deployed on the platform of choice. Finally “live” measurements from the executing solution are fed back to Solution Provisioning so that the solution’s actual achievement of the desired business goals can be fully certified. Variances from the expected results may result in solution refinement or new inputs to the SO Business Improvement discipline.

Improvements to the Enable Track

In order to provide a more complete process framework, which runs right through to solution operation, the enable track has been enhanced as follows:

• Solution/Service Platform Architecture: To design both the solution and the service infrastructure in terms of available platform hardware and software. (Previously this activity was confined to services as part of the Service Portfolio process unit; we now broaden the approach to encompass solutions).
• Solution/Service Deployment: To install both solutions and services in their target run time environments according to their specifications. (Previously this activity was confined to services as part of Service Operations and Management; we now recognize deployment as significant in its own right and broaden the approach to encompass solutions).
• Also, note that the remaining enable disciplines have been broadened to include solutions as well as services platform design and installation, and operations and management. In particular Solution/ Service Operation and Management incorporates the measurement and runtime control of the solutions (as well as services) with respect to their specifications and makes available solution and service execution metrics to solution provisioning to certify Let’s take a more detailed look at the new solution disciplines and the overall interplay of these disciplines with other disciplines in the provide and enable tracks.

Solution Architecture and Design

The discipline of Solution Architecture and Design links the disciplines of Service Oriented Business Improvement (SOBI) and Service Oriented Architecture and Design (SOAD). A Solution Architecture comprises a Business Process Architecture and a System Architecture, aligning business process activities with the functions and services of the underlying systems. This alignment clarifies the relationships between business processes and systems leading to a better understanding of the scope and impact of proposed business process changes. In the end, the goal is to employ services for all business process activities that appear in multiple business processes or that embody logic coordinating other activities.

This solution architecture and design process represents a refinement over previous work. A key assumption in the earlier work was that a business process consisted entirely of the orchestration of a set of services, with very little regard to non service-oriented software. The revised approach allows for the broader case where any type of software component may be involved in the business process, which of course represents cold reality for most organizations

Solution Architecture and Design takes place in two distinct process units. The first, Synthesize Solution Architecture, defines the total solution architecture (business processes and systems). The second, Solution Design, consists primarily of creating the descriptions for the solution components and ensuring consistency between these descriptions.

Synthesize Solution Architecture

The total architecture approach represents something of a sea change in that solution architects are refining their efforts based upon iteration at both the business level (for example, with business process modeling) and at the technology level (for example, with deployment modeling).

Synethesize Solution Architecture begins with an initial breadth-first inventory of all the business processes that appear to be impacted by the project. This inventory is based on the business goals set forth in the project charter (an output from SO Business Requirements Planning), and is the first cut at defining the full scope of the project. A minimal amount of data is gathered in order to rank the business processes according to their perceived level of design difficulty and business importance.

Once the initial ranking is complete actual architecture synthesis begins. Starting with the most challenging (highest ranking) business process, the architecture of each business process is explored. Constraints on the business process design are identified as follows:

• The interactions of this business process with the rest of the enterprise’s business processes are defined.
• To the extent that the business process already exists and is just being modified, the existing business process is modeled and the interactions between the portion that will be modified and the portion that will remain fixed are defined.

Taking into consideration the project charter constraints (the quantified benefit expectations and associated cost and schedule constraints), potential architectures for the business process are explored. Each architecture defines the participants (human and system), their activity responsibilities, and their required interactions. This exploration occurs in two phases, the first leaving the “system” (at least the new functionality portion) as a single black-box participant in the business process and the second exploring the structure of the black box. Services are identified at the notional⁵ level – both the use of existing or proposed services and the concepts for new services. Once a satisfactory architecture has been found for the most challenging business processes, other business processes in the inventory are explored until the complete solution architecture has been defined.

The focus of this process unit is on identifying the business process participants (both human and system), their responsibilities with respect to individual business process activities, and the required interactions between the participants. Various combinations of participants and activity assignments are tried and evaluated until a suitable architecture is identified.

Two important inputs to Synthesize Solution Architecture are the IT inventory and the catalog of existing services. Where available, these are augmented with the SPP that lays out the plans for services. To the extent that business objectives are not significantly compromised, the solution is architected to utilize existing services. For proposed services in the SPP, the solution architecture contributes the solution’s requirements for the service. In the case of an existing service that does not exactly satisfy the business objectives, the cost of service modification is weighed against the cost of compromising the business objective, and a decision is made as to whether the service should be modified. Should modification be required, a service requirement is generated indicating the required modifications.

Design Solution

The design of the solution details its architecture, and consists primarily of creating the descriptions for the solution components and ensuring consistency between these descriptions. The data structure generated by one component, for example, must match the data structure on the consuming component. For components that already exist but do not have rigorous specifications, the description is used as the basis for communicating the requirements to the component owner and obtaining a commitment to satisfy those requirements. For components and services that are already well specified, the descriptions are used to determine whether the existing component or service is suitable.

Where new services are required, or when notional services already exist in the SPP, service requirements must be specified in sufficient depth to enable service descriptions to be developed (within SOAD). For efficiency on both sides, it is a best practice for these service descriptions to be developed jointly with the team that will ultimately be responsible for the architecture, design, and provisioning of the service.

It is important to note that in addition to working out the software components and services required to achieve the expected business improvements (identified as part of the Solution Project Charter), the discipline of Solution Architecture and Design also involves:

• defining the required collaboration between organization units required to implement the business process(s)
• scouting ahead to ensure technical feasibility of the solution by examining possible solution deployments to different platforms.

Once the Solution Architecture has reached a reasonably stable state through several iterations it is necessary to scope the solution into a number of Solution Design Scopes for input to the Service Provisioning discipline; see below.

Coordinated Architecture Scope

The total architecture approach creates an architecture through an iterative refinement process. The development and reuse of services is a key part of this approach that requires collaboration between the solutions architects and the services architects. This is where the idea of coordinated architecture scope comes into play (Figure 3), by establishing those elements of the service oriented process framework that are involved in this iterative refinement. This coordination ensures that solution architecture is in balanced harmony with both the (enterprise) service architecture and the solution/service platform architecture. It is a way of managing risk. For example you might have the most perfect solution and set of services but if they do not deploy comfortably to the available platforms, this will all be for naught. Conversely you might have developed a highly robust platform architecture in the technical labs, but again this will all be for naught if the services have not been correctly identified, or if the solution does not address a real need for business improvement!

Figure 3: Coordinated Architecture Scope

The Coordinated architecture scope involves a number of activities highlighted below. It is recognized that these activities call for a further process unit to be incorporated within the SOA Management discipline, as it works at “meta level” across the tracks.

Assessing the Wider View

As the Solution Architecture and Design progresses, so requirements for services emerge. These requirements set forth the service characteristics that are essential for the solution but must also be assessed with respect to the wider enterprise view embodied in:

• Project charters produced by IT Strategy and Architecture that identify expected IT project costs and benefits, including the projected return on investment (ROI) from proposed services to be produced and reused by the projects.
• The Service Oriented Business Plan (SOBP) that is a result of the discipline, Service Oriented Business Requirements Planning.
• The SPP for the entire enterprise or a part of the enterprise, where this is available. Where an SPP is not available the service requirements must be evaluated with respect to the broader context and the SOA policy refined as part of a Project Service Plan.
• The Service Catalog, where this is available
• The IT Inventory
• Other outputs from IT Strategy and Architecture such as enterprise architecture models.

As mentioned earlier, service requirements may be met by reusing existing services. Some of these services may already be available as part of the SPP or (where an SPP does not exist) at least be documented within a service catalog.

Other service requirements identified by the Solution Architecture and Design discipline may be met by new services. These services must be documented as part of the Project Service Architecture, or as part of an SPP Fragment where an SPP is available.

Planning Legacy System Usage

It is also important to weigh up service requirements against the Legacy Transition Plan to determine if any of these requirements could be met by either reusing an existing service (typically an underlying service in the form of a wrapper) or by “servicizing” the legacy functionality. The alternative, attempting to directly employ the legacy functionality, may turn out to be more complex (and expensive) than placing a service wrapper around it, particularly if other business processes require access to the same functionality.

Planning Deployment

Deployment must also be a consideration. You must figure out whether the solution architecture is technically feasible by examining how the services and the other software components comprising the solution are to be deployed. Technology operating constraints play a major role here, and their consideration is the genesis for the newly added discipline of Solution/Service Platform Architecture. The estimated cost of implementation and deployment must also be determined, as each solutions project operates under cost and schedule constraints.

Reassessing Scope

Each total architecture iteration evolves across the tracks of the SO process. Having driven the architecture towards deployment, you then return to see whether the proposed solution architecture achieves the business process improvements set forth in the solution project charter (a result of the SO Business Improvement activity). The identification of services with potentially broader utilization scope may result in proposals to widen the project scope to include an analysis of these other usages. Excessive costs or schedules may result in proposals for narrowing the project scope. For example, you might have specified a service with additional operations to support anticipated future utilization, but given the cost and schedule constraints you may have to scale down the service requirements to just those operations required for the present solution.

Solution Provisioning

The Solution Provisioning discipline addresses the following needs:

• to plan the sourcing, usage, testing and overall management of solutions
• to specify and design the individual solution components
• to ensure that solutions actually achieve their purpose.

This discipline recognizes the fact that solutions, no less than services, must be provisioned. In fact, the two disciplines of Solution Provisioning and Service Provisioning work in analogous fashion at different levels of abstraction and in a cyclic fashion.

Solution Cycle 1: Plan to Assemble/Implement

The first part of this cycle – plan to assemble/implement (see figure 4) - takes place as follows. The Solution Design Scope, Solution Architecture, and Component/Service Descriptions are input to the Solution Specification Design and Coordination set of process units, which break down as follows.

The Plan Solution process unit:

• Plans how the solution is to be designed and used, and identifies suitable providers. So, for example, just as we determine whether services are to be developed or run in house or by third parties, so we do the same for the components of the solution.
• Determines how the overall solution will be integration tested, operated and managed. The process unit generates the integration test plan (an order of assembly), systems validation test plan (testing the overall system for compliance with the Solution Architecture and Design, particularly with respect to SLAs), and the user acceptance test plan.
• Creates a Solution Profile that recommends the level of detail required in the component specifications that are to be implemented. The level of detail may include a high-level design of non-service components. The level of detail is based upon the business risks associated with the operation of the solution itself and the business risks entailed in the association between the implementing parties and the group sourcing the components.

The Design and Specify Solution Components process unit:

• Specifies and designs the non-service components required to support the solution. These specifications contain the level of detail indicated by the Solution Profile
• Interacts with Service Provisioning as it develops service specifications. These specifications are developed within Service Provisioning in a process of iterative refinement as the Solution Provisioning specifies and designs the non-service components that are specific to this solution. A key aspect of this refinement is to check Service Specifications and SLAs (supplied through the Service Catalog by Service Provisioning) to determine the optimum solution design given available services. Of course we are only focusing on the parts of solution design that are particularly relevant to enhancing the SO process right here. In addition there will be a range of further activities involved in solution design that may stretch from user interface design to internal coordination logic.

The resulting Component Specifications (for the solution-specific components) are input to the Solution Implementation/Assembly discipline to develop the solution specific components. The components thus created, along with the required services, are then assembled and tested to produce a fully assembled and tested solution, along with a Solution Implementation Design. At this point the actual automation units that will realize the services may not be available, so stub service interfaces may be employed by the solution assembly. End-point systems, depending on their availability, may have to be stubbed out in the early integration test as well. Note, who actually does the solution assembly depends on the providers identified in the Solution Profile.

Figure 4: Solution Cycle 1: Plan to Assemble/Implement

Solution Cycle 2: Assemble/Implement to Deploy

A second cycle – assemble/implement to deploy – now takes place as shown in figure 5. Once the services to be used in the solution have been certified and published (within the Service Provisioning discipline) and deployed (by the Solution/Service Deployment discipline), Solution Assembly/Implementation fully integration tests the solution.

Figure 5: Solution Cycle 2: Assemble/Implement to Deploy

Solution Cycle 3: Deploy to Run

The third cycle– deploy to run – is shown in figure 6. The Solution Implementation Design and the Tested Software Solution (from Solution Assembly/Implementation) are now input to Solution Certification set of process units which include a Certify Solution process unit. This process unit includes the following activities:

• System Validation Testing which checks the solution meets its requirements as an integrated assembly across its complete constituency of different consuming business processes and associated users. Testing that the combined elements of a solution assembly meet their individual performance goals is not enough. Communications latency and the emergent behavior of collaborative interactions (think vehicles bunching on a busy highway) mean that the whole system must be exercised to demonstrate that the business process goals are being met. System Validation Testing focuses on validating that the components comprising the solution were, in fact, built to specification and that when combined the resulting behavior is as desired.
• User Acceptance Testing, which tests the integrated solution with respect to use in the actual business environment.

In the case of “in-house” deployment once the user acceptance testing has been completed and the results accepted, the tested Solution is forwarded with a Services Deployment Authorization and Solution Operation Level Agreement to the Solution/Service Deployment discipline. The Deployed Solution is then forwarded for execution to the Service/Solution Operation Management discipline. The solution is now in operation.

Figure 6: Solution Cycle 3: Deploy to Run

Solution Cycle 4: Run to Improve

Once in operation, a final cycle takes place. Data is collected and analyzed to measure whether the expected business benefits are actually being realized. Ideally, the solution should be controlled in realtime according to its specification. In practice, feedback to Solution Certification (in the form of a Review Solution Certification process unit) and thence to SO Business Process Improvement is necessary as shown in figure 7. Refinement and tuning of the solution, including iterating the design and implementation of individual components and services, may be required. This data collection and analysis continues until one of two events occurs: either the analysis concludes that the business benefits have, indeed, been realized, or the business concludes that additional investment in refining and tuning the solution is no longer warranted. A conclusion that the business benefits are being realized constitutes the certification of the solution.

Figure 7: Solution Cycle 4: Run to Improve.

Putting it all Together

Figure 8 brings together the various pieces that we have discussed in this article within the context of the simplified process framework presented earlier (see figure 1).

Figure 8: The Enhanced SO Process Framework: Detailed View

For those with access to a larger format printer (A3), or who donŐt mind taping two pieces of paper together, Figure 8 is also available as a large poster suitable for printing

Click here to view larger version

Key Deliverables

The following deliverables have been introduced as a result of the CBDI – TIBCO collaboration.

Solution Architecture
Business Process Architecture + System Architecture

Business Process Architecture

The structure of the business process, indicating who the participants are (both human and system), their functional responsibilities, and the interactions between them. The architecture also describes how non-functional requirements will be met (e.g. throughput, response times, etc.). System participants are represented at a very high level in this architecture.

System Architecture

The structure of the systems components involved in the business process, indicating the components, their functional responsibilities, and the interactions between them. The architecture also describes how the non-functional requirements for the system components (both service and non-service) are derived from the non-functional requirements of the business process.

Component Description

A description of a component based on the systems architecture. This description covers the externally observable behavior required to support the business process. It identifies all required interfaces (without detailing them) and all data structures required (again without detail). It describes the non-functional requirements that must be met by the component.

Component Specification

A refinement of the component description now driven to complete detail. Each interface is specified, along with each data structure. The specification may contain a high-level design as well.

The Solution Delivery disciplines are listed in table 1, together with sample process units with deliverable definitions. Note that SO Business Requirements Planning and SO Business Improvement (that are not covered in this article) are included for completeness.

Table 1 – Solution Delivery Disciplines (Examples)

Concluding Remarks

This article has focused on the shift from pure service consumption to fully architected solution delivery. The confines of an article do not permit elaboration of all the artifacts and various nuances of the enhanced SO process. In addition we have limited our attention specifically to business process improvement (as opposed to business products), and excluded areas such as fast track evolutionary development and capability modeling. Neither have we discussed how the process maps on to organizational structures and roles.

However we do believe this is a significant step forward especially in improving the interworking and collaboration between SOA efforts and software development projects, and in providing a much more practical context for SOA efforts that is related to the business needs that drive software development projects

Looking to the future we hope that this article also provides a platform for more detailed work in defining the various process units, tasks, techniques and artifacts that comprise a full blown process. Both CBDI and TIBCO are very keen to work with our customers to evolve this advice to meet the challenges and leverage the opportunities that customer organizations face every day.

Paul Allen is a principal consultant, author and educator with Everware-CBDI International. Paul is widely recognized for his innovative work in component-based development, business-IT alignment and service-oriented architecture. His detailed knowledge stems from over 30 years experience in the management and development of large-scale business systems, and combines with a uniquely practical understanding of the problems companies face as they apply new technologies in search of business value. Paul is a widely published author whose current focus of interest in helping organizations transition to service orientation is detailed in his most recent book - Service Orientation: Winning Strategies and Best Practices - published in April 2006.

Dr. Paul C. Brown is a principal software architect at TIBCO Software Inc. and the author of Succeeding With SOA: Realizing Business Value Through Total Architecture and the upcoming Implementing SOA: Total Architecture In Practice. His model-based tool architectures are the foundation of a diverse family of applications that design distributed control systems, process control interfaces, internal combustion engines, and NASA satellite missions. Dr. Brown’s extensive design work on enterprise-scale information systems led him to develop the total architecture concept: business processes and information systems are so intertwined that they must be architected together. Dr. Brown received his Ph.D. in Computer Science from Rensselaer Polytechnic Institute.

1. The Service Oriented Process, Allen, P., CBDI Journal, February, 2007
2. Succeeding with SOA: Realizing Business Value Through Total Architecture, Brown, P., Addison Wesley, 2007
3. Service Orientation: Winning Strategies and Best Practices, Allen, P., Cambridge University Press, 2006
4. Note that solutions are certified at a level commensurate with risk in terms of integration testing and user acceptance testing prior to, and subsequent to, live running, to ensure that they meet the business goals targeted. Further certification may take place in the shape of regular reviews.
5. A notional service is a capability offered by a provider to a consumer according to a contract. “We might not know whether the service should be implemented as software, or whether it is a manual service, or it’s some combination of the two. But we do have the notion of a Service so we can create an instance of Service (notional) and assign that service a name” - Learn by Example: A Populated SAE Meta Model, Dodd, J., CBDI Journal, February, 2007
6. A Business Feature can be a business process, business capability or business product
7. The Solution Project Justification typically includes models of the redesigned business feature; the models used are likely to be adaptations of existing models, such as business process models, used in a particular organization.

  Images used in CBDI Journal November in Powerpoint format (Gold subscribers only)

  CBDI Journal November in Adobe PDF format (Gold subscribers only)

Reports in November Journal

• Main Menu
• Editorial
• Architected Solution Delivery: Enhancing the Service Oriented Process
• Web 2.0 and Enterprise Architecture
• SOA for the Business Developer: By Margolis and Sharpe
  SOA and WS-BPEL: Composing Service-Oriented Solutions with PHP and ActiveBPEL
  By Yuli Vasiliev

Print this page