A Framework for Integrating the Publishing Workspace

A Framework for Integrating the Publishing Workspace

E–Business Application Integration WorkshopObject Management Group, Orlando Florida

Niwot Ridge Consulting, 2001 1


Glen B. Alleman

E–Business Application

Integration Workshop

Orlando, Florida

January 29th – February 1, 20001

Prepared by Niwot Ridge Consulting, 2000In association with Net–Linx Publishing SystemsSacramento, California

Today we’re going to talk about how software is being constructed in a different way for the publishing industry. This different way is the use of component technologies and CORBA as the means of assembling these components into a complete system.

The motivation for this effort is not technology alone, it driven by the fundamental changes in the business processes of newspapers, magazines, and other news sources.




What’s Happening in the Industry

Enterprise integration should be viewed as a strategic initiative for every large enterprise. The complexity of and the demands on modern computing environments requires a systematic, centralized approach to integration requirements to achieve optimal value for the totality of enterprise information assets. Message brokers are an emerging class of products that can be increasingly applied at the enterprise level.

— Patricia Seybold Group

It’s the value of the technology assets that drives business decisions. How can the value of these assets be increased in the presence of changing standards, increasing complexity, increasing competition, and the Internet? One answer is the better use of commercial off the shelf components and reuse of these components once they have been integrated into a system.




Macro Economics and Software

♦ In a Harvard Business Review paper, Brian Arthur [†] outlined the differences between: – Increasing returns and

– Decreasing returns

[†] “Increasing Returns and the New World of Business, W. Brian Arthur, Harvard Business Review, 74(4) Jul-Aug, 1996, pp. 100–110.

♦ In the Decreasing Return world, Optimizationis the critical factor, characterized by:– Bulk processing and repetitive processes

♦ In the Increasing Return world, Adaptation is the critical factor, characterized by,– Market instability

– Multiple potential outcomes

– Unpredictability

The Internet age has brought change to nearly every industry that uses software. In publishing the Internet means multi-media, re-purposing, new standards, and new architectures for the underlying systems. The economics of the Internet Age mean an increasing returns business strategy, instability of markets, multiple outcomes, unpredictability, and the ability to lock a market. These changes have been seen in many market segments, cable TV, cellular telephones, financial services and health care. Adapting software to this new economy requires the same processes as adapting a business, flexibility, and the ability to operate in the presence of chaos.




Macro Economics and Software

♦ In this Increasing Return environment, adaptation requires:– Agility

– Flexibility

– Incrementalism

– Rapid development

– Loosely defined interfaces

– Operations within chaotic environments

– Managing work–state rather than workflow

♦ All of these attributes can be found in component based software systems

The Increasing Returns economy requires adaptability rather thanoptimization. Adaptability means agility, flexibility, incrementalism, rapid development, interfaces that are not formally defined, chaotic business and product environments, managing the state of the business rather than the flow of the business. Component software systems provide the tools and architecture to meet these requirements. Traditional software tools and architecture do not.




Components are IT Plumbing

Object Request Brokers

Models

Inheritance

Polymorphism

EncapsulationObjects

n-Tier

3-Tier

2-Tier

Architecture

Java the Language

IDL

IIOPDCOM

OMG

Java Beans

Components

Java the OS

Client Server

The WebFault Tolerance

Load BalancingRemote Access

XML

eXtremeProgramming

SmallTalk

UML

RAD

In information systems, business components are like plumbing, they connect all the important pieces of the system, provide the flow of important resources, and keeps everything going in the right direction at the right time, for the right reason.

One plumbing model is the interchangeability of parts in the North American Standard plumbing fixtures. Parts and pieces can be assembled into a final system by simply connecting each component in the proper manner to form the completed system.

This may not be the experience of the home repair person, after the 3rd

trip to the hardware store for that “special” part, but it is the norm in the commercial plumbing world.




Component Based Systems

To succeed in this era, software development organizations must come to terms with the

realities of the market.

What are component based systems? Why are they different and better than traditional integrated systems? How can the user understand the differences and make an informed decision about the technology strategies needed to move into the next generation of publishingsystems?




Component Market Imperatives

I’ve got a Great Idea!

Here’s howthis will help

everyone

Define the componentsin terms of the existing architecture.

We can add this to the next release and everyone will be happy

The system has new featuresand all the old features still work

The power of components can be seen in a example of system development. The developers have built a component based infrastructure on which commercial applications can be assembled into a product. Someone (who will remain a nameless marketing executive) discovers a new requirement for a new component or subsystem that must be added to the system without disrupting the other application components. In the component environment the interfaces between the components provides the necessary isolation to allow a new application to be added with minimal impact. The business rules and softwareprotocols shared by all components form the foundation of this process.

The fundamental concept here is the market place drives this process. The marketing department will have significant input to the outcome, but the application domain and the system behaviors are controlled by external forces, not the traditional product marketing forces.




The Market Drives Architecture

♦ Systems developers must accommodate marketplace instabilities.

♦ Component come and go and often do not integrate well with each other, even from the same supplier.

♦ If integration is achieved at all, it often breaks down when new version of a component is introduced.

♦ The traditional, tightly coupled, systems also have an n2–n connections, versus nconnections for a component based system with bus connectivity, like CORBA.

Many of today’s technology decisions are driven by the market, not the vendor. The internet, CORBA, communications networks, programming languages, databases, system architectures, are all defined external to the business needs of the user. Adapting to these standards requires an architectural vision, not just a product plan. This vision must include the evolving standards as well as the evolving component technologies provided by external suppliers. Building “purpose built” applications moves away from this vision and must be considered carefully.




Systems Must Accommodate Change

♦ New components are desirable for their innovative features

♦ New components necessary in response to market pressures

♦ The downside is that new components may not be compatible with each other or with older components

♦ Some means of integrating the new components, while maintaining the old is needed

♦ This method is based on managing the interfaces between components rather the content of the components

Change is constant. Change is disruptive. Change is the engine that drives innovation, increasing returns, market share, product displacement, product obsolescence.

In order the survive in the presence of change the underlying system architecture must not only support change, but must thrive on change –that is the cost of change is not related to the size or complexity of the application domain, but is a function of the change. This is a linear change cost model, rather than a non-linear (exponential or greater) models found in API based implementations.




Compressed Processes Are The Norm

♦ Requirements, analysis, design, and implementation cycles must be compressed and commingled.

♦ Often the only way to know a component works is to try it.

♦ Previously discrete activities are often indistinguishable.

♦ This may appear as chaos but it is “Just in Time Competency,” though highly focused prototyping.

♦ In the end it is all about “Time to Market.”

The traditional processes of analysis, design, development, and deployment have difficulty dealing with rapid change, adaptive markets, and changing market requirements. Component based systems provide the means to develop product in the presence of these forces.




Component Suppliers

♦ Systems depend on component suppliers, not just components. These suppliers provide parts and building blocks, as well as tools.

♦ This regime is new and is similar to traditional manufacturing supply chain management.

– Suppliers of parts linked to suppliers of subassemblies

– Subassemblies assembled into assemblies

♦ Unlike manufacturing, the standards are set by the market place, not the final assembler

– CORBA

– Java and Swing

– Database technologies

In the Best of Breed market, the suppliers not only provide a product, they also provide a collection of products and the environment in which they operate. These collections of components may include items from other suppliers. This situation adds to the complexity of integrating components and increases the importance of system architecture, component isolation, and industry standards in assembling the final product.




Controlling the Fit, Form and Function

♦ Designers must remove the sources of “misfit” among components.– Misfit between user needs and the available components

– Misfit among components that cannot be seamlessly integrated

♦ Mediation and negotiation are the keys to the first misfit.

♦ Bridging, wrapping, integrating, and federating are the keys to solving the latter misfit.

♦ This process is called “strategy”– Strategy is creating fit among a company’s activities [†]

– This depends on doing many things well – not just a few

[†] “What is Strategy,” M. E. Porter, Harvard Business Review, 74(6) pp. 61–78.

The success of a component based strategy depends on controlling the fit, form, and function of the individual components as well as the final product. This is done using “system architecture.” There are many acceptable architectures for component based systems, picking one and deploying it becomes the challenge of the development organization. Maintaining the system architecture is the next challenge. In the component based approach the architecture can be maintained by isolating the individual components from each other. This isolation allows each application domain to perform its function without impacting other domains. Shared information, state, and status is then provided by a “clearing house” approach that maintains the separation, while “federating” the individual components. As a result, components can be added or removed with minimal impact on other components.




Technology Competence

♦ A critical asset to the vendor is the technical competence of the staff

♦ Technical competence begins and ends with good people and managers

– The once lowly software engineer has emerged as a dominant player in the success of the organization

– Technology skills are changing as rapidly as the technology itself.

– No one can stand still for long

– Skills of the past are obsolete, skills of the future have yet to be defined

– Change is constant

– Welcome to the Internet

In order to design, develop and deploy component based systems, new skills are needed. These include Object Oriented analysis and design, formal system architecture, understandings of fault tolerance, scaling, and n-tier distributed processing. These skills are the foundation of the internet. Searching for a product and vendor with these skills is a critical success factor in the acquisition of any modern system.




Why The Alternatives Have Problems

♦ Rapid development requires “Separation of Concerns,” which is not found in tightly integrated systems.– Component interfaces prevent tight coupling

♦ Fault Tolerance requires dynamic reconfiguration.– CORBA load balancing, naming services, and dynamic object

management

♦ Platform independence comes with Java– CORBA is both Java and C++ based

– CORBA can be a 100% Pure Java environment

The alternatives to component based systems can be found in the market place. They include traditional client server based systems, best of breed applications connected to centralized databases, and tightly integrated applications using programmatic interfaces. Each of these approaches fails to address the fundamental issues with modern software systems such as, adaptive configurations, isolation of functions, layering of components, heterogeneous integration strategies, and standards for integration.




The Bottom Line on Components

♦ The difference between components and traditional systems may appear complex, but…

♦ The bottom line is components provide:– Interchangeable parts

– Scalable systems

– Architectures that are adaptable to changing market needs and new product releases

♦ All of this translates into– Faster time to market

– Lower sunk cost for the development process

– Higher flexibility in a changing business and technical marketplace

The difference between component based systems and more traditional system integration approaches lies not in the techniques but in the results of the process. The outcome of the integration in the component architecture is distinctly different than an integrated system based on traditional API’s. Flexibility, isolation, separation of concerns, portability across languages, platforms, operating systems, and locales is provided “for free” with the CORBA architecture.

All of these attributes must be built from scratch in the non CORBA implementation.




The World of Publishing Systems

In the Publishing World, there are fundamental applications that are needed to

get the paper out.

In the publishing domain, change is coming somewhat late compared to other businesses. This is an advantage that can be used to avoid many of the past difficulties.




The Component Domain

Advertising

Output

WorkflowGraphicsPictures

Pagination

Wire Service Editorial

Using a Best of Breed approach has several consequences. It is unlikely that the Best of Breed application has been built as an OEM product, that is intentionally built to be used “within” another application. Many BoB products have API’s but these API’s make the assumptionthat the application is the center of the process. This is called “Inversion of Control” and takes place when one application is integrated with others, and all the applications assume they are in control.




Federation Principles

♦ The individual domains do not need to know the details of each other’s semantics, data formats or organization– A shared object repository connects

the domains

Advertising

Output


Pagination


– Data Objects carry information about state, status and location in the object repository.

– Processes and events are synchronized using CORBA’s publish and subscribe mechanisms.

♦ Content created in one domain remains there until needed in another domain– This is the basis of a Federated system.

In our system these Best of Breed components have no specific knowledge of the internal workings of other domains. This “federated” approach results in a unique architecture with several “bookable” benefits not normally found in a traditional “integrated” system.

§ The individual system components are not directly coupled to each other through API’s, removing the “coupling” issues found in tightly integrated systems.

§ The semantics of the data in each domain remains isolated until it is transferred to another domain. At that point the receiving domain is responsible for performing the translation of the syntax and semantics (if necessary). XML is used in most cases, so this translation process can be further simplified.

§ The transfer process is event driven and “pull” oriented. This further decouples the domains, since the logic of when the content is needed is in the destination domain, nearest to point of use.




Federation Principles

♦ A connector object joins application domains to the object store and other services– On the domain side, the connector

invokes create, read, update, and delete methods for each application domain supplied through an API

Advertising

Output


Pagination


– On the UNA side, the UNAC use UNA methods for transferring information between the domains, using NewsGrams.

♦ The benefits of CORBA are inherent in the UNA, including– Load balancing

– Scalability

– Platform independence

A connector, the Universal NewsGram Architecture Connector (UNAC) provides the isolation and wrapping of the domains. The UNAC provides:

§ CRUD semantics for each domain.

§ Exception handling

§ Initialization and restart

§ Identification and notification of new capabilities




The UNA Connector (UNAC) Object

♦ A three layered pattern is used– Interfaces present a common face to the world

– Syntactic mapping hides the domain specific data formats

– Semantic mapping hides the domain specific commands

♦ The application is wrapped is a portable object that isolates the content from the metadata

ApplicationSpecificAdaptor

Code

ApplicationDomain

DomainSpecific

Code

MethodInterfacefor thisUNAC

Method_Name

Method_NameMethod_NameMethod_NameMethod_Name

Method_Name

Method_Name

SemanticMapping

SyntacticMapping

InterfaceAdapter




Best of Breed Components

♦ Integrating Commercial Off The Shelf (COTS) software is now the norm– Using purpose built software has very little economic appeal for a

vendor or end user

– Interchangeability of the software, people, and formats providesthe agility needed to deal with change

– software becomes the enabler of the economic outcome

♦ Putting these components together allows– Focus on the “business” of business

– Open systems, that mean “plug and play” software components

– Re-capitalizing of the investment in software and hardware

♦ This means gaining and holding a competitive edge

Overcoming this Inversion of Control issue as well as other control integration issues requires an architecture designed to isolate the applications while also providing state, status, and data integration.




The Result of this Approach

♦ A systematic process is created to add or change functionality without impacting existing components– The news business is an environment where change is the

norm.

– At the same time legacy systems continue to operate for decades.

LegacyApplications

EnterpriseApplicationIntegrationFacilities

NewlyDeveloped

Applications

PackagedApplications

A NewBusiness

Application




Components Build Systems

Using a component strategy, systems are constructed in the same way other

“component based” products are built.

Once a component strategy has been selected, the assembly of these components into a system is driven by the business architecture not the technical architecture.




The Manufacturing Paradigm

DigitalAsset

Manager

Adobe InDesign™Pagination Layout

System

NewsGramObjectStore

LotusDomino™

Groupware System

Story

Page

Image

ContentManagement

Reporters

GraphicsArtist

Paginators

ContentProcurement

Creation ofContent

Assignment ofContent to the

Publication

ContentAssembly

ContentDelivery

OutputManagement

Publishing thePaper to

Specific Media

Here’s a simple picture of the business processes involved in publishing a newspaper. The content creation and management within each domain can be provided by a best of breed application. The connectivity between the domains is provided through a layered set of objects which reside in a central clearing house. The state and status of the content in each domain is communicated other to domains using this meta object. The content remains in the application domain until it is assembled on the published page for output.




EAI Deployment Strategies

Application to

ApplicationIntegration

Eve

nt

Exc

hange

Data

base

Exc

hange

Purpose of The EAI

Update another domain Read another domain

MessagingSystems

Publish andSubscribeSystems

TransactionProcess MonitorSystems

WarehousingAnd DatamartSystems

This bridging process can take several forms.

There is database centric forms and event exchange forms.

For each form there are different behavioral results.

The Insiight™ system makes use of data federation and publish and subscribe.




Why is EAI Important to Publishing?

♦ Integration issues transcend the boundaries of point solutions.– Single vendor solutions are no longer acceptable

– New classes of applications now dominate the market• Sales force automation

• Web based customer interaction

• Process driven environments – workflow

• External content acquisition

• Re–purposing of output to non–traditional markets and devices

♦ “Processware” is a class of software that sits between different applications and allows those applications to interoperate in a transparent manner.

In the publishing business the content of the publication is usually located in a variety of systems. The days of a single monolithic system that does everything for everyone are over. Today’s systems are in fact systems of applications, each with a special capability. The users of this system want each of these specialized systems to work together seamlessly without requiring any changes to the underlying data or processes in each domain.

This transparent interoperation of COTS products is what distinguishes the EAI solution from the integrated solution. This is what separates Insiight™ from any other product that tightly couples COTS into a system.

It is the loose coupling of components along with the data and process neutral architecture that defines EAI Federation.




What Drives EAI?

♦ Complex application integration – Multiple languages used to implement systems

– Multiple operating systems

– Network protocols

– Proprietary and standard application integration interfaces

– Multiple application development and management tools

♦ All of these different components create a barrier to the seamless integration of the enterprise

♦ EAI is the mechanism to connect publishing components as well as ERP components such as SAP and PeopleSoft.

Since there are several sets of terms here, EAI, CORBA, Components, let’s focus on the EAI aspects. EAI is the overarching concept of deploying components. CORBA forms one of the foundations for thecomponent technology used to implement EAI. EAI results in a seamless integration of a variety of system applications to form a meta system. The Insiight™ system is one example of an EAI based product. There are others of course, but in the publishing business EAI is an emerging concept. With the introduction of ERP systems, the connection between content creation and publishing and the business side of this process can be made.




A Look at the Details of the UNA

Possessing a CORBA based architecture is a necessary but not sufficient condition for

success in the distributed object business




The Big Picture

Text management

Graphics managementAd placement

and issue structure

Pagination

UNAC:Editorial

UNAC:Pagination

UNAC:DAM

Domino (Text Mgmt) Oracle, LDAP, db’s, user profiles, DAM)

Scalable Clustered Data Servers

IssueManager

Insiight Application Services ArrayJava/CORBA server processes, running on smaller, distributed platforms (Windows NT or Sun Solaris), providing scalability and high availability.

……

Create text – reportersReceive text – remote and wireCopyflow – routingOrganization – header dataNewsroom – reporter assignsCollaboration – email

& – messagingCommunication – calendar

– contacts

CORBA

Receive images – remote and wireOrganization – desks, basketsLo-res/High res – OPI Conversion – image formatsColor mgmt – calibrationValidation – preflight

Create content – reportersReceive content – public, advertisers, remote and wireLayout – template mgmt Organization – site mgmt

Space res. – ad bookingPage dummy – ad geometryIssue structure – create the Issue ManagerAd content – location specified in run sheet

Create pages – deadline or future Flow text – from any text storePlace graphics – from thumbnailsManage styles – native toolsCopyfit – same H&J engineJumps – manual or autoProductivity – shape tool and tools

shape factory

UNAC:Graphics FS

UNAC:WebPub

PurgeManager

NOSManager

Security Manager

QueryManager

EventManager

CORBA-based, n-tier middleware

Online publishing




Text managementdomain

Graphics management Ad placementand issue structure

Pagination

Query the graphics management system for lists of thumbnails and descriptive information on photos and graphics in the digital assetmanagement system.

Drag and drop a thumbnail onto a page, and Insiight places the lo-res preview with OPI comments.

Insiight creates the issue structurefrom an ad reservation system likeALS. Opening a page in the layout tool also brings up the page template normally assigned to the page, and any ads reservations for the page. If the ad graphic is finished and available on the DAM, the graphic itself will come up placed on the page.

Work with the pagination toolof your choice – Adobe Indesign or QuarkXpress, with their companion products for copy fitting; Adobe InCopy or Quark CopyDesk

Query the text management database for lists of stories based on content or category. XML-generated lists are based on query templates. You can re-sortor change field order on the fly.

Then drop the text onto a page, or open a text editor to change copy.Edited copy is saved back to thetext management domain

Pub Group

Title

Date

Edition

Zone

Section

Page

Newshole

The Insiight Workspace hosts Internet Explorer, and any browser-based tools for working with your web site.

Online publishing

The Deliverable to the User




The Users View of the System§ Unified desktop puts data resources

next to applications

§ Configurable action buttons lets users add queries or launch applications

§ Queries are user configurable for different domains and selection criteria

§ All windows are resizable

§ Preferences are saved centrally

§ Tabbed application area can run multiple programs concurrently

§ Applications can be expanded to full screen

§ Content caddy – a private folder structure for shortcuts to pages, stories, graphics, saved centrally

§ Content caddy segments can be sent to other users and automatically appear in their content caddies

§ Drag and drop from lists to application area for most applications

§ Drag and drop from lists to content caddy segments

§ Call up metadata (header info) by right clicking on a list item

§ Edit metadata from list, i.E., Change basket value to route stories or graphics

§ Metadata form (header) can be displayed and edited in the application area, above the editor

§ Different editors can be assigned to assets, or launched automatically by double clicking the list item

§ In the future, InSiight will incorporate new editors for other assets – adobe Photoshop, go live, dream weaver, illustrator, others

§ Instant messaging reports who is logged on, converts to email if addressee not on system

§ Multiple lists can display in list viewer

§ Lists can be resorted and reordered on the fly

§ Columns can be resized or hidden




CORBA is an EAI Enabler

EAI is a broad architectural solution space. CORBA is one paradigm to build EAI systems. CORBA is well suited to the

publishing industry, as well be shown next.

The middleware market is an enabling technology market. This is different than an integration technology using API’s, or a technology that allows items to be connected (RMI for example). This software is independent of the applications and at the same time is the glue that holds them together.




What Were The EAI Alternatives?

♦ MQ Series – Windows, AIX and Mainframe based implementations

– Message Oriented

♦ Microsoft Message Queuing Systems– Similar to MQ Series but Microsoft-centric

♦ Third Party Products– NEON

– Vitria

– Saga Soft

♦ All are EAI oriented but there are other Critical Success Factors to consider

There are always alternatives to every technology, here are somealternatives to CORBA.




Critical Success Factors

♦ Adaptive platforms– Unix and NT dominate here, but there are multiple flavors of

UNIX as well as databases and there is always the Mac

♦ Agile application selection– The “Best of Breed” list changes with every generation of

product

♦ Scalable resources– Scalable in terms of performance, capacity, users, locations

♦ Robust (fault tolerant) architectures– Tandem like behaviors are the benchmark

♦ Natural transition to the Internet– CORBA and Java “are the net,” anything less will become an

“Instant Legacy” system

These Critical Success Factors could be applied to any system integration architecture. So what makes CORBA unique here? CORBAprovides these in one package, external from the business applications. It provides them through a set of standard interfaces, a standard processing model, and a standard set of platforms and programming languages.




Staying Focused on the Outcome

♦ The bottom line is Business Issues usually dominate Technical Issues– So the choice of a technology must be

based on a business analysis

– What is the cost of providing a feature set using a specific architecture?

♦ Buy or build a monolithic set of applications– How will these be integrated?

– How will the scale to the internet?

– How will these applications adapt to changes in the Best of Breed product list?

♦ Adopt the CORBA (or other middleware) paradigm

All technical decisions need to start with the business process. By focusing on the business requirements the technical aspects of the decision can be put in the proper perspective.




Some Analysis of the Economics

[†] ”Dynamic Object Technology – Introduction,” R. Laddaga, CommunicationsOf the ACM, 40(5), 1997, pp. 38–40.

[‡] The Intranet: Slashing the Cost of Business, I. Campbell, International DataCorporation, 1996.

♦ CORBA and Object Technologies are specifically designed to support rapidly changing software with cost proportional to the change, rather than the size of the application. [†]

♦ Well documented studies show ROI well over 1000%, with three themes: [‡]

– Rapid deployment of heterogeneous platforms

– Widespread acceptance due to ease of use of browser based technologies

– The realization of the promise of openness, and the ability to replace components at will.

The economics of component architectures is becoming better understood. Although this topic is still being developed, here are some results from the industry. It is important to understand that these results are not marketing and sales figures, they come from refereed journals.




Conclusions




Market Forces Control the Outcome

♦ Rapidly changing business and technology environments.

♦ Increasing competition from a variety of sources– Consolidation

– Internet

– Other media – cable, DBS

♦ Increasing dependence on Commercial Off The Shelf (COTS) products– No one want custom products, they want customizable

products.

– No wants to wait, they want customization now.

The economics of successful software development and deployment depends on an “increasing return” model that can address the business and technical needs of the market. These needs have been presented in the previous slides. They are:

1. Environmental independence

2. Local transparency

3. Separation of the interface from the function

4. Self describing interfaces

5. Platform binary independence – this is a Java capability

6. A universal Application Component Framework

There are several alternative middleware architectures that can address these needs, but CORBA provides one of the best standards that is universally hosted.

Documents

A Framework for Integrating the Publishing Workspace