Business Integration Principles
Table of Contents
Overview
Business Integration is a specialised competency centre within IT Enterprise Architecture whose focus is satisfying core, value-chain IT business requirements that transcend vertical domains and isolated business functions.
In general, such core business requirements tend to originate from B2C, B2B, and BPM-related business functions. A B2C function provides products such as Web Portals, IVR Systems, and Kiosk applications. Typical examples of B2B functions are invoice printing, logistics and ERP reconciliation. A BPM function is independent from Business Integration but benefits from automation services provided by it: for example, the automatic disablement of an employee’s badge when they leave the company.
A key aspect of Business Integration is that it is a facilitating function—in general, it is seldom a profit centre. Its goal is to produce benefits elsewhere in the organisational structure.
For example, a B2C “Track my order” iPhone application would typically require integration with the CRM, Billing and Logistics domains in a relatively large (10,000+ employee) organisation. The default approach to deal with a problem like this is called Ad-Hoc Integration. In Ad-Hoc Integration, integration needs emerge directly from requesting projects and are satisfied in a project-specific fashion. What Business Integration attempts to do is to provide a more efficient—cheaper and faster—means to deliver such products that cross cut through various domains.
Some organisations mistake Ad-Hoc Integration with Point-to-Point Integration and embark on a dogmatic SOA crusade which results in a Hub-and-Spoke (ESB-centric) model. This model may still be an Ad-Hoc Integration model unless adequate governance processes have been introduced and effective SDLCs and BPDLCs have been established. As a matter of fact, it is possible to have an efficient—well governed—Point-to-Point Managed Integration model which is superior to an inefficient SOA/Hub-and-Spoke Ad-Hoc Integration one.
In the rest of this section, we address the difference between Business Integration and other related topics:
- Business Integration and BPM: BPM is more mission critical than Business Integration.
- Business Integration and Technical Integration: technical integration is a pre-requisite of every fully functional system.
- Business Integration and SOA: SOA is a possible tactic to achieve Business Integration.
Business Integration and BPM
Business Process Management (BPM) is a core business competency centre which is, in principle, independent from IT. Such a competency centre may rely on whiteboards, pens and Post-it notes technology. The introduction of a BPMS constitutes merely an enhancement over a less efficient technical capability. However, what is important to keep in mind is that introducing a BPMS does not lead to the formalisation of a BPM Competency Centre (CC) if one does not already exist.
The fact that automated business processes may be instrumented as APIs creates confusion since a BMPS is often employed as a software implementation tactic (over other possible choices such as Java, .Net, and so on). The SOA dogma is partially responsible for this confusion since it suggests a strong link between an abstract concept, such as “orchestration”, and a concrete implementation tactic such as WS-BPEL. Thus, organisations feel compelled to roll out a BPEL engine and actually use it so that the ghost of “lack of service-orientation” can be scared away.
A Business Integration department may well “own” a BPMS but, in our opinion, it should be restricted to the implementation of mid-level business processes. Core business processes are too critical to be owned by a Business Integration. For this reason, we make it clear in our reference illustration that the BPM CC should not depend on the Business Integration’s BPMS capability; if this competency centre happens to possess one at all.
Moreover, there is a key difference between human-only processes and processes that may be instrumented as APIs. The latter require all the assurances typical of the software world: static code analysis, test code coverage, profiling, unit testing and so on.
Business Integration and Technical Integration
Business Integration is always a form of technical integration. The difference is in the human interpretation of each integration type. Business Integration fulfils a direct business need which is normally understood outside the IT domain. This is why the focus of Business Integration is serving B2C, B2B and BPM functions.
Conversely, even though technical integration is ultimately supportive of business needs, the relationships are only understood by IT. For example, any relatively large system has dependencies such as database management systems, directory services, storage systems, name servers, and so on. Furthermore, some systems complement other systems. In this case, the technical integration required to achieve such complementation is intrinsic to the solution and constitutes a technical type of integration too—for instance, a CRM system that integrates with a Cisco communications system to capture call interactions.
Business Integration and SOA
Service-Oriented Architecture is an approach to Business Integration based on the belief that integration points are called services and that they should exist outside the front-end and back-end systems so that they can be instilled with so-called SOA Principles—abstraction for example. In most cases, this implies the need for middleware—namely an ESB.
Many ideas that emerged under the SOA banner are applicable in the wider Business Integration context and the need for fat middleware should not be automatically assumed. Nowadays, many capabilities (such as security and throttling) can be provided by using appliance technology which is cheaper, more reliable and does not involve managing and supporting a complex Operating System/Application Server/RDBMS stack.
Logical Layers
A logical layer is a set of conceptual boundaries that help group, visualise and understand aspects of an Enterprise IT Architecture. From a Business Integration perspective, three broad layers are worth considering:
- Front-end Layer: this layer is primarily constituted by bespoke applications that make the organisation unique in the market. The bulk of front-end applications tend to be of the B2C nature (Web portals, Smartphone Apps, and so on). In second place, there are B2B, Cloud Applications and Process Interaction interfaces.
- Business Integration Layer: this layer provides consolidated and managed access to IT resources. The key gateway to such resources is the Service Catalogue.
- Back-end Layer: the supply chain which is constituted by all vertical domains, discrete applications and cloud/B2B back-end services.
In addition, we consider four additional layers that cross-cut through the front-end, business integration, and back-end layers:
- Business Intelligence: Gathering, consolidation and reporting on business data. Business Intelligence may allow real-time reporting by using enabling BMPS technology.
- Security & Quality of Service (QoS): Authentication, Authorisation, SLAs, OLAs, etc.
- Operations: Monitoring, Management & Support.
- Governance: Business Governance, Runtime Governance, Design-time Governance.
We will now look at each of the layers in more detail.
Front-End Layer
The front-end layer is constituted by human-facing applications (such as web portals, smartphone & tablet apps and IVR systems) and edge computer systems such as B2B consumers and cloud applications. It encompasses all the actors that are furthest away from the business' core and integration systems. Most of their features are centred on interaction–use cases rather than discrete business logic.
Front-End Layer systems are categorised in Internal Applications, B2C Applications (Consumer-Facing) and B2B/External.
Internal Applications
Internal applications are used exclusively by a business' employees, consultants, and selected suppliers. They typically require access credentials and are only accessible through an intranet or a VPN.
Examples: Timesheets, Customer Care, Expenses Submission, Human Workflow & Business Process Interaction interfaces.
B2C Applications
Customer-facing applications are either publicly open to a wide audience (such as the company’s homepage) or to future/paying customers (such as password-protected customer areas of a website). B2C applications are typically complex systems consisting of multiple sub-layers and components.
Examples: Company portal, self-care portal, smartphone app, tablet app, SMS update.
B2B/External
Business-to-Business applications do not necessarily constitute an orthogonal category with respect to internal, and B2C applications. To be strict to the definition, a B2B application would be, for example, an invoice printing company which accesses the business integration layer to fetch invoice records. However, today many applications that run in the cloud actually serve internal and B2C use cases as well.
What all B2B and external front-ends have in common is that they are only partially governed by the business' authorities.
Examples: Cloud CRM application, Invoice Printing, Third Party App.
Business Integration Layer
The business integration layer’s purpose is to facilitate access to enterprise-wide, cross-domain data and functionality to the front-end layer. This layer is divided into two broad sub-layers: the Service Layer and the Business Automation Layer.
Service Layer
This is the layer concerned with API and data augmentation. The purpose of this layer is to add capabilities to the functions provided by the back-end layer.
Core capabilities are:
- Routing: the identification and routing of messages to the appropriate back-end systems.
- Delegation: the use of “broker” systems to perform activities that are too inefficient or inconvenient for the front-end systems to perform.
- Normalisation: the homogenisation of data when its format varies across systems. This capability’s complexity may range from a simple mapping technique to a fully-fledged canonical data model.
- Caching: the persistence of back-end data that changes infrequently and/or whose changes are inconsequential.
- Adaptation: the exposure of back-end systems that use “obscure” protocols using an interface which is more efficient or convenient for front-end systems.
Business Automation Layer
The business automation layer complements the service layer in two modes:
- Business logic in this layer may consume services from the service layer.
- Business logic in this layer may be exposed as services through the services layer.
The Business Automation Layer is not a single platform but a family of platforms that host business logic that is enterprise-wide and cross-domain. Most typical platforms in this layer are:
- BPMS: If the business has a BPM practice.
- Customer Enabler Platforms: (Specific for each language)
- J2EE Application Cluster
- .NET/IIS Application Cluster
- BRE Engines
In the case of a BPMS system, it is worth remembering that a “service façade” to a business process is merely a way of automating the triggering (and/or change the state of the process). Some processes may be of human interest only; some others may be useful in an automation context only.
The Service Catalogue
The service-catalogue represents all the IT resources that can actually be leveraged in the organisation.
Our interpretation of service is that of managed API. A service is not necessarily an artefact created inside an ESB. Therefore, a service catalogue should contain all APIs that can be leveraged and whose location and life cycles are known.
If front-end teams need to resort to back-end SMEs to figure out the resources that exist—for instance, because the Business Integration team only takes care of ESB-trapped services and disregards everything else—then dogmatic SOA principles are probably in action. Business Integration’s first and foremost goal is to reduce integration friction by consolidating, and making accessible, all of the supply chain’s relevant knowledge—barring applicable security and accessibility constraints.
Back-End Layer
The back-end layer is a general boundary that captures the whole of the supply chain that is visible from a Business Integration perspective. It typically encompasses all discrete IT systems and vertical domains.
In FTSE100/Fortune 100 organisations, domains may present more IT complexity than an entire medium-size business. When this is the case, a “domain” has, in itself, a front-end layer, a business integration layer and a back-end layer, consisting of multiple specialised subsystems.
Examples
-
Small business: Back-end applications: QuickBooks, SAP Business One, SalesForce (Cloud), MySQL/Microsoft SQL Databases with customer data, Bespoke VB.NET Billing application, etc.
-
Medium-sized business: Domains: CRM, HR, ERP, Billing. Each of them running a vertical software clustered platform with separate database servers and identity providers. They are typically front-ended with a firewall/load balancer and access to each domain application requires custom credentials.
-
FTSE100/Fortune100 business: Conventional vertical domains (CRM, HR, ERP, Billing, etc) plus various composite and specialised domains (for example, Fraud Prevention and Business Innovation). Each domain may be, in itself, a complex IT landscape consisting of front-end, integration, and back-end layers. The integration layer may deploy ESBs and possess substantial amounts of bespoke business logic to overcome the limitations of shrink-wrapped software’s out-of-the-box features.
Cross-cutting Layers
The main broad cross-cutting layers are Business Intelligence, Security & Quality of Service, Operations and Governance.
Business Intelligence
Business intelligence and data warehousing practices typically collect data from ultimate remote data storage systems (mainly RDBMSs). However, many business data may be collected in real-time as processes run and APIs are invoked. In the BPMS world, BAM provides real-time business intelligence without the need of using the batch-based ETL model.
Security and Quality of Service
Although platforms used to rely on a black-box security model, security and quality of service augmentation is increasingly being implemented in a cross-cutting fashion by means of identity providers, distributed enforcement points and central certificate authorities.
Operations
Operations, as a cross-cutting IT function, is concerned with the day-to-day running of IT systems and the support to their users. For this purpose, a wide range of technical capabilities are required such as:
- Logging
- Monitoring
- Alert systems
- Incident and Ticket Managements
- Support Lines: L1/L2/L3
- Backup and Restore Systems
Governance
Governance usually starts at the executive board level (CIO), and then falls into the Enterprise Architecture realm until it cascades to specific domains. In certain organisations, domains that are profit centres may have more “governance power” than enterprise-wide practices (such as BPM); therefore, such a hierarchical model needs to be taken with a pinch of salt.
As far as Business Integration is concerned, governance is broadly divided into two aspects: Design-time Governance and Runtime Governance.
Design-time Governance
Unless otherwise specified, “governance”, on its own, implies design-time governance. Design-time governance looks at what should be built, how, and why—what is the business justification? Likewise, it looks at what has been built in the past, and what lessons have been learned. Design-time governance is perpetually centred upon baseline (current) states and target states but it is not interested in the “now”; for example, whether a service is up or down.
Runtime Governance
Runtime governance is the one which concerns the live architecture, the architecture that is running right now, and not that which will be implemented in the future. For example, if a consumer has exceeded a service’s capacity due to unpredicted demand, it is a runtime governance’s duty to discuss the SLA with the consumer’s owner.