UNIVERSITY OF WATERLOO
Software Engineering
AAnn EEvvaalluuaattiioonn ooff WWoorrkkffllooww FFrraammeewwoorrkkss ffoorr tthhee
AAuutthhoorriizzaattiioonn ffoorr EExxppeennddiittuurree SSyysstteemm
Xyz Inc.
Sometown, Somewhere
Prepared by
Some Student
Student ID: 20000000
User ID: sstudent
3A Software Engineering
September 5, 2006
Some Student My Street My Town, ON A1A 1A1 September 5, 2006 Dr. William Wilson, Director Software Engineering University of Waterloo Waterloo, ON N2L 3G1 Dear Dr. Wilson:
The enclosed report, An Evaluation of Workflow Frameworks for the Authorization for Expenditure System, is my third work term report. It is related to my work at Xyz Inc., where I was employed during my 3A work term.
Xyz is a consulting firm that specializes in custom software development. I was
employed as a consultant, and worked on several client-facing projects over the course of the work term. One such project involved the design, development, and deployment of a human workflow solution for Client Company, a leading stuff producer.
One of the fundamental design decisions that I faced during the planning phase of
the project was choosing a workflow framework. This report evaluates two alternatives against multiple criteria and identifies the more appropriate choice for the project.
I would like to thank Dr. Andrew Morton and Santheeban Kandasamy for
suggesting the use of Multi-Criteria Decision Analysis (MCDA). I would also like to thank my colleague, Colleague, for his insight into the application of Microsoft BizTalk Server to human workflow systems.
I hereby confirm that I have received no help, other than what is mentioned
above, in writing this report. I also confirm this report has not been previously submitted for academic credit at this or any other academic institution. Sincerely, Some Student Student ID: 20000000 Encl.
iv
Executive Summary
Client Company, a leading stuff producer, has engaged Xyz Inc., a consulting
firm, to design a human workflow solution to automate their expenditure approval
process. A key deliverable for the planning phase of the project involves a
recommendation of a workflow framework. This report provides a comparison of two
alternatives, Microsoft BizTalk Server 2006 and Microsoft Windows Workflow
Foundation (WF), assessing their suitability to the system.
This report evaluates the alternatives against a set of seven criteria: persistence of
long-running workflows, transparency into the workflow, flexibility for human-driven
processes, maintainability of business rules, extensibility, risk, and cost. Because of the
subjective nature of the criteria, this report leverages Multi-Criteria Decision Analysis
(MCDA) to provide a robust, quantitative analysis of the alternatives.
The two alternatives are similar in some aspects and very different in others.
BizTalk and WF have similar persistence and business rules mechanisms. BizTalk, as an
enterprise-class product, provides a more robust set of tracking tools, while WF is
considerably more flexible for human workflow. WF, which is freely distributable, is
much less expensive than BizTalk, but this factor is offset by the fact that, as pre-release
technology, WF is more risky as a technology investment.
Ultimately, the analysis indicates that WF is the more appropriate solution,
whether or not the cost factor is included. This report recommends that the architect
design a solution based on WF, that the development team undergo whatever training is
necessary, and that the project manager allocate additional time for deployment.
v
Table of Contents
Executive Summary ........................................................................................................... iv List of Figures .................................................................................................................... vi List of Tables .................................................................................................................... vii 1 Introduction............................................................................................................. 1 2 Background............................................................................................................. 3
2.1 The AFE Workflow .................................................................................... 3 2.2 Design Constraints ...................................................................................... 4 2.3 Evaluation Criteria ...................................................................................... 5
2.3.1 Persistence....................................................................................... 6 2.3.2 Transparency................................................................................... 7 2.3.3 Flexibility........................................................................................ 7 2.3.4 Maintainability................................................................................ 8 2.3.5 Extensibility .................................................................................... 8 2.3.6 Risk ................................................................................................. 8 2.3.7 Cost ................................................................................................. 9
2.4 The Analytic Hierarchy Process ................................................................. 9 2.4.1 Overview......................................................................................... 9 2.4.2 Criteria Weighting ........................................................................ 10
2.5 The Alternatives........................................................................................ 11 2.5.1 Microsoft BizTalk Server 2006 .................................................... 11 2.5.2 Microsoft Windows Workflow Foundation.................................. 11 2.5.3 Alternatives Not Considered......................................................... 11
3 Evaluation ............................................................................................................. 13 3.1 Persistence................................................................................................. 13 3.2 Transparency............................................................................................. 13 3.3 Flexibility.................................................................................................. 14 3.4 Maintainability.......................................................................................... 14 3.5 Extensibility .............................................................................................. 15 3.6 Risk ........................................................................................................... 15 3.7 Cost ........................................................................................................... 16
4 Results................................................................................................................... 17 5 Conclusions........................................................................................................... 19 6 Recommendations................................................................................................. 20 References......................................................................................................................... 21 Acknowledgements........................................................................................................... 22 Appendix A: The Analytic Hierarchy Process.................................................................. 23
vi
List of Figures
Figure 2.1: The AFE Workflow.......................................................................................... 4 Figure 2.2: Sequential Workflow........................................................................................ 5 Figure 2.3: State Machine Workflow.................................................................................. 6 Figure 4.1: Graphical View of Results ............................................................................. 18
vii
List of Tables
Table 1.1: Authority Amounts ............................................................................................ 1 Table 2.1: Rating Scale ..................................................................................................... 10 Table 2.2: Criteria Weighting ........................................................................................... 10 Table 4.1: Weighted Scores .............................................................................................. 17 Table A.1: Rating Scale .................................................................................................... 23 Table A.2: Criteria Matrix ................................................................................................ 24 Table A.3: Normalized Criteria Matrix ............................................................................ 24 Table A.4: Comparison Matrix ......................................................................................... 25 Table A.5: Normalized Comparison Matrix ..................................................................... 25 Table A.6: Final Scores..................................................................................................... 25
1
1 Introduction
Client Company is an international stuff doing company headquartered in
Toronto, Ontario. With the recent acquisition of A Company Inc., Client Company is
currently the world’s largest stuff producer [1]. The company operates 27 things across
five continents and trades on the Toronto, New York, London, Euronext, and SWX Swiss
stock exchanges.
Client Company’s spending policy mandates that an Authorization for
Expenditure (AFE) form is submitted for all cash outlays greater than or equal to
$10,000. Once submitted, an AFE is routed to a series of approvers. The AFE is
escalated up the organizational hierarchy until it is approved by an individual with
sufficient authority. An employee’s authority amount is determined by her position
within the company, as shown in Table 1.1*.
Table 1.1: Authority Amounts
Position Authority Amount
Chief Executive Officer (CEO) >$100X
Chief Operating Officer (COO) $100X
Designated Senior Executive $50X
Vice President $20X
General Manager – Category A Things $10X
General Manager – Category B Things $5X
General Manager – Category C Things $2.5X
General Manager – Category D Things $1X
Stuff does not have any system in place to automate the AFE workflow; AFEs are
routed by e-mail, by fax, or in person, while approved AFEs are archived in filing
cabinets. The client has identified several issues with this manual process:
* The absolute authority amounts are confidential, and have been replaced with relative authority amounts.
2
• Once an AFE is submitted, there is no transparency into the state of the
workflow. Often, AFEs accumulate on an approver’s desk while other
employees attempt to locate them.
• There is no electronic repository of approved AFEs. The filing cabinets
contain thousands of AFEs, and finding a specific one is extremely time-
consuming.
• Often, due to human error, an AFE is not routed to the correct approvers,
and does not obtain approval from an individual with sufficient authority.
• As a result of the various inefficiencies in the manual process, it takes far
too long to obtain approval for an AFE.
Client Company has engaged Xyz Inc., a consulting firm, to design, develop, and
deploy a software solution to automate the AFE workflow. One of the key deliverables
for the planning phase of the project is an evaluation of workflow frameworks and a
recommendation for the AFE project.
This report provides an evaluation of two workflow frameworks, assessing their
suitability to the AFE system. It first provides some additional background on the AFE
workflow. Next, it defines a set of evaluation criteria and provides a brief overview of
the alternatives. The report then evaluates the alternatives against each criterion. Due to
the subjective nature of the criteria, the report leverages Multi-Criteria Decision Analysis
(MCDA), a methodology for providing a robust evaluation of alternatives against
subjective criteria. Finally, the report summarizes the results of the analysis, draws
conclusions, and makes a recommendation.
The reader should have some background in enterprise software development.
Familiarity with MCDA is recommended but not required, as an overview of the
particular approach taken is provided in Appendix A.
3
2 Background
2.1 The AFE Workflow
The first step in the workflow involves the initiator preparing and submitting an
AFE form. Once an AFE is submitted, the system must analyze its contents and
determine the sequence of approvers; the business rules that determine this approval
chain are outside the scope of this report. The AFE is then routed to the first approver in
the approval chain. This approver receives an alert that her approval is required, reviews
the AFE, and submits her decision and comments. Each approver has the option to
approve or decline the AFE. If approved, the AFE is routed to the next approver in the
chain. If declined, the AFE is routed back to the initiator, who must revise and resubmit
the AFE. Once an AFE has been approved by each individual in the approval chain, it is
marked as approved, and the workflow is complete. Figure 2.1 provides a simplified,
conceptual model of the AFE workflow.
4
Initiator completesAFE form
Initiator submits AFE
Approve?
Final Approver?
Yes
No
Complete
Yes
No[route for revision/resubmission]
Initiator Route to next approver
Approver reviews AFE
Start
Figure 2.1: The AFE Workflow
In addition to the requirements described above, the system must be flexible to
accommodate exceptions to the normal process. As well as being able to approve or
decline an AFE, an approver should have the option to delegate to an alternate approver.
Furthermore, the system administrator should have the option to alter the approval chain
at runtime; for example, she may wish to bypass certain intermediate approvers and route
the AFE directly to the CEO.
2.2 Design Constraints
Client Company has specified the following technical constraints for the AFE
system:
• It will be deployed to a single server.
• It must run on Microsoft Windows Server 2003.
• It must use either Microsoft SQL Server 2005 or Oracle Database 10g.
5
The workflow frameworks evaluated by this report adhere to these technical constraints.
2.3 Evaluation Criteria
Workflow can be classified into two types: system workflow and human
workflow. The AFE process is a typical example of the latter. The two types are
fundamentally different and present different challenges.
In general, the participants in system workflow are software applications. The
integration of disparate software applications is often referred to as Enterprise
Application Integration (EAI). If this integration connects multiple businesses, the
workflow is often referred to as Business-to-Business (B2B) workflow. System
workflow typically executes in a rigid, predefined sequence. The process, rather than the
participants, dictates the sequence of actions. System workflow is often modelled as a
sequential flowchart, such as the one illustrated by Figure 2.2.
Figure 2.2: Sequential Workflow
6
As the name implies, the participants in human workflow are people. Although a
human-driven business process may leverage software applications, it is the human
participants that drive the workflow, not the software applications themselves. Human
workflow typically executes in a flexible, dynamic fashion, where participants often have
a choice of actions. The participants, rather than the process itself, dictate the sequence
of actions. This type of workflow is often modelled as a state machine, such as the one
illustrated by Figure 2.3.
Figure 2.3: State Machine Workflow
The following subsections define the criteria against which the alternatives are
evaluated. The criteria are specific to the AFE system, but most human workflow
systems have very similar high-level requirements.
2.3.1 Persistence
Unlike system workflow, human workflow must be long-running and persistent.
Whereas a system process can take seconds, minutes, or hours, a process that involves
human tasks can take days, weeks, or even months. Furthermore, system processes often
execute from start to finish without interruption, whereas human-driven processes
typically execute in short bursts of processing separated by comparatively long periods of
idle waiting. The workflow framework must support long-running workflows by
persisting their state to a permanent data store and unloading them from memory when
they enter an idle state.
7
2.3.2 Transparency
A system process often executes in one batch and yields one of two outcomes:
success or failure. In contrast, a human-driven process transitions among a series of
states until it ultimately reaches the acceptance state. The workflow may idle in a given
state for extended periods of time while it waits for human intervention. Businesses need
transparency, or visibility, into the state of the workflow. Indeed, one of the most critical
problems with Client Company’s existing manual process is the lack of transparency into
the state of the workflow.
2.3.3 Flexibility
Whereas system processes are generally rigid and predefined, human-driven
processes tend to be more flexible and dynamic, due to the inherent differences in the
way machines and humans operate. In system workflow, the process dictates what the
participants (i.e., the software) must do, whereas in human workflow, the inverse is true;
the participants (i.e., the individuals) dictate what the process should do.
One way to achieve this flexibility is to replace the sequential model of execution
with one based on the concept of a state machine. A state machine is event-driven, and
does not have any predefined path of execution. For example, depending on an
approver’s decision, the AFE workflow could take totally different paths of execution.
When an AFE is approved, the workflow could transition from the “Awaiting Approval”
state to the “Approved” (acceptance) state. Conversely, when an AFE is declined, the
workflow could instead transition from the “Awaiting Approval” state to the “Awaiting
Revision” state. Upon revision and resubmission of the AFE, the workflow could then
transition back to the “Awaiting Approval” state. As the number of possible events
increases, the workflow quickly approaches an infinite number of potential paths of
execution. At the same time, designing the workflow using a sequential model of
execution becomes increasingly difficult.
Another way to increase flexibility is to allow the workflow to be modified at
runtime. New workflow instances can be created from a template, but diverge from the
8
template at runtime by adding, removing, and rearranging steps as necessary. For
example, AFE workflow instances could start with a default approval chain based on
business rules, but allow approvers to be skipped at runtime to accommodate exceptional
circumstances.
2.3.4 Maintainability
A system process is typically data-driven; it receives some input, performs some
processing and auxiliary tasks, and returns some output. In contrast, human workflow is
often driven by a set of complex business rules. As businesses change, these rules can
change as well, requiring modification of the workflow.
Many human workflow systems have business rules implemented in code, which
is costly to maintain, as changes to business rules require code changes. Other human
systems have business rules defined in the application’s relational database, resulting in
overly complex database schemas. A robust human workflow framework should provide
dedicated storage and maintenance of business rules. The current trend is to use some
type of human-readable markup language supplemented by a visual rules editor.
2.3.5 Extensibility
Human workflow systems often involve many of the same activities. For
instance, the submission-approval pattern exemplified by the AFE workflow is an
extremely common scenario. That said, each human-driven process has its own unique
characteristics. For this reason, it is insufficient for a workflow framework to simply
provide a library of common activities. These activities must be extensible by supporting
the addition of custom code.
2.3.6 Risk
As with any software platform or library, the choice of workflow framework
involves some degree of risk. Many companies prefer to use a well-established product
with proven success in similar systems. On the other hand, a leading-edge workflow
9
framework may provide a superior solution, but at the cost of increased risk. Newer
technology is often perceived as immature, and sometimes undergoes drastic changes in
successive versions. Pre-release software in particular is often subject to feature changes
and application programming interface (API) refactoring.
2.3.7 Cost
Workflow frameworks range from free, open-source libraries to custom-built,
million-dollar platforms, with varying degrees in between. Client Company has stated
that cost is unlikely to be the deciding factor, unless the discrepancy is exorbitant. For
the most part, a cheap, inferior solution should not be favoured over a more expensive,
superior one.
2.4 The Analytic Hierarchy Process
2.4.1 Overview
The subjective nature of the evaluation criteria makes it somewhat difficult to
provide a robust, objective analysis of the alternatives. In recognition of this challenge,
this report leverages MCDA, a management sciences methodology that uses simple
mathematics to analyze complex decisions [2]. More specifically, this report uses the
Analytic Hierarchy Process (AHP), an MCDA approach published by mathematician
Thomas Saaty in 1980 [3]. AHP transforms a set of qualitative comparisons into a set of
quantitative scores. A weighted objective function consolidates the scores for each
alternative into a single, useful metric. For the sake of brevity, the AHP calculations
themselves are not included in the body of the report, but are instead outlined in
Appendix A.
After establishing the criteria and alternatives, the next step in AHP is to define a
rating scale that converts qualitative comparisons into numerical values that can be used
in calculations; this scale is illustrated in Table 2.1.
10
Table 2.1: Rating Scale
Factor Subjective Rating
1 Equally preferable
2 Slightly preferable
3 Significantly preferable
4 Very preferable
5 Extremely preferable
2.4.2 Criteria Weighting
The analyst uses a rating scale similar to the one above to quantify the relative
importance of the criteria. The result is a set of weightings for the criteria, as shown in
Table 2.2.
Table 2.2: Criteria Weighting
Criterion Weighting
Persistence 24%
Transparency 24%
Flexibility 24%
Maintainability 10%
Extensibility 6%
Risk 6%
Cost 6%
A full justification of these weightings is outside the scope of this report. Xyz has
rated the relative importance of the criteria based on discussions with Client Company.
11
2.5 The Alternatives
2.5.1 Microsoft BizTalk Server 2006
BizTalk Server 2006 is an enterprise server application that automates business
processes and integrates disparate software applications. BizTalk was originally
designed for EAI and B2B processes, but businesses are increasingly using it to automate
human workflow as well. The current version is the fourth release of the product, and is
an evolutionary update to the previous version, BizTalk Server 2004.
2.5.2 Microsoft Windows Workflow Foundation
Windows Workflow Foundation (WF) is a component of Microsoft’s next-
generation developer platform, the .NET Framework 3.0, which will be included in
Windows Vista and ported to Windows XP and Windows Server 2003. WF’s goal is to
provide a unified framework for both system and human workflow. Microsoft is
targeting WF not only for third-party developers, but also for use in future versions of its
own products, including the 2007 Office System and the next version of BizTalk [4].
2.5.3 Alternatives Not Considered
Although BizTalk was met with immediate success in the EAI and B2B market, it
was criticized for its shortcomings when used to automate human workflow. Microsoft
responded to this criticism by including Human Workflow Services (HWS) in the 2004
release. In brief, HWS was not well-received and did not see widespread adoption; as a
result, BizTalk’s workflow engine will be replaced by one built on WF for the next
release of the product, at which point HWS will become a deprecated technology. Since
HWS will eventually become unsupported, Xyz cannot consider its use for the AFE
project.
While this report considers two Microsoft technologies, there are a number of
third-party, enterprise-class workflow platforms available, such as SourceCode’s K2.net,
Captaris’ Workflow, and Ultimus’ BPM Suite. The problem with these alternatives is
12
that they are prohibitively expensive; the licensing for many of these platforms can cost
tens or even hundreds of thousands of dollars. Client Company is not prepared to make
such a large investment solely for the AFE project, and there are no immediate plans to
automate other business processes. While Client Company has not set a maximum cost
for the project, they have left consideration of technology investments to Xyz’s
discretion.
13
3 Evaluation
The following subsections provide a direct comparison of BizTalk and WF
against each criterion. The AHP calculations transform the subjective ratings into
weighted, numerical scores.
3.1 Persistence
Both BizTalk and WF provide a built-in persistence mechanism, and do so using a
very similar approach: workflow state is serialized and stored in a dedicated SQL Server
database. With respect to persistence, BizTalk and WF are equally preferable.
3.2 Transparency
BizTalk excels in this area through a feature called Business Activity Monitoring
(BAM), which allows workflow milestones and key performance indicators (KPIs) to be
logged to a SQL Server database. BizTalk Server 2006 introduces several new features
that leverage BAM, including the BAM Portal, a web-based monitoring tool, and the
Alert Manager, which can subscribe to workflow events and automatically send out e-
mail alerts [5].
Like BizTalk, WF provides a tracking service that uses a SQL Server database;
however, WF provides only the tracking service itself, and does not implement any
delivery channels for it. The developer can use a reporting framework such as SQL
Server Reporting Services to provide the functionality of BizTalk’s BAM Portal, and a
custom notification service to provide the functionality of BizTalk’s Alert Manager.
While BizTalk and WF both provide a base framework for workflow tracking,
BizTalk provides built-in delivery channels for the information that a WF developer
would have to implement. For this reason, BizTalk is slightly preferable to WF.
14
3.3 Flexibility
BizTalk’s origins as a systems integration platform are evident in its lack of
flexibility. This shortcoming is the main reason why its workflow engine is being
deprecated in favour of one built on WF in the next version of the product. BizTalk
workflows must be designed in a sequential fashion; there is no support for a state
machine model of execution. In addition, workflows must be strictly defined at design
time, and are immutable at runtime.
WF was designed from the start to provide flexibility for human workflow.
Workflow designers have a choice between a sequential and a state machine model of
execution. Furthermore, either type of workflow can be modified at runtime using an
API dedicated to dynamic modification. With respect to flexibility, WF is very
preferable to BizTalk.
3.4 Maintainability
In BizTalk, a business rule is implemented as a condition and a set of actions, and
represented as an IF-THEN statement. The Business Rules Engine defines rules using a
declarative language called XLANG, an implementation of Extensible Markup Language
(XML), and stores them in a dedicated SQL Server database. The Business Rule
Composer serves as a visual rules editor. Having defined a set of rules, the workflow
designer can insert a Policy object to load and evaluate the rules at a specific point in the
business process.
WF’s Rules Engine is very similar to BizTalk’s Business Rules Engine. Like
BizTalk, WF represents business rules as IF-THEN statements; as a convenience, WF
developers can also include an ELSE clause. Rules are defined using Extensible
Application Markup Language (XAML), another XML-based markup language, and are
stored in a dedicated rules file. The RuleSet Editor and Policy object are very similar to
their counterparts in BizTalk. With respect to maintainability, BizTalk and WF are
equally preferable.
15
3.5 Extensibility
While BizTalk provides a comprehensive library of common activities, these
activities are sealed, and cannot be supplemented with custom code. Similarly, although
the workflow as a whole is ultimately compiled into a class, there is no provision for the
developer to include her own custom code in this class. These limitations aside, it is
possible for the developer to invoke method calls into external assemblies through the use
of the Expression object and the associated Expression Editor, which allow the inclusion
of short code snippets.
WF also provides a library of common activities, but many of these activities can
serve as base classes for custom activities. In addition, WF uses a partial class model to
allow for easy addition of custom code. The workflow designer generates a markup file,
while the developer writes custom code in a separate file; the compiler then consolidates
the two files into a single class. The workflow designer can include a Code object to
invoke calls into the custom code at a specific point in the business process. With respect
to extensibility, WF is significantly preferable to BizTalk.
3.6 Risk
BizTalk is an established product, in terms of both its role in Microsoft’s line of
offerings and its use by businesses worldwide. The 2006 version is the fourth release of
the product, and Microsoft has committed to at least two future versions. Furthermore,
customers can expect support for at least the next decade.
In contrast, WF is pre-release technology, and is currently in the Release
Candidate (RC) stage. There is no guarantee that Microsoft will not introduce breaking
API changes between now and the release date, or even that WF will be released at all.
The former is the primary concern, but the technology is fairly close to release, and the
fact that the last two RC versions have not introduced any breaking API changes is an
encouraging sign. The latter, although possible, is fairly inconceivable given not only the
16
late stage of the project, but also the fact that WF is a key component of the .NET
Framework 3.0, the next version of Microsoft’s developer framework.
Although the risk associated with WF seems fairly low, the fact remains that
BizTalk is an established product and WF is pre-release technology. As a result, BizTalk
is significantly preferable to WF.
3.7 Cost
BizTalk Server 2006 Standard Edition is licensed at $8,500 (US) per processor.
Thus, Xyz has estimated the cost in Canadian currency and including all taxes to be
$10,846 for a single-processor server or $21,692 for a dual-processor server. The actual
price may vary depending on any existing licensing agreements between Client Company
and Microsoft, but the above figures provide an approximate estimate suitable for
comparison purposes.
In contrast, WF is a freely redistributable component of the .NET Framework 3.0,
and does not require payment of any licensing fees, even for use in a production
enterprise system. With respect to cost, WF is extremely preferable to BizTalk.
17
4 Results
The criteria weightings and the subjective comparisons serve as the parameters to
the AHP calculations, whose results yield a set of weighted scores. Table 4.1
summarizes these weighted scores, while Figure 4.1 provides a graphical representation
of the same data.
Table 4.1: Weighted Scores
Criterion BizTalk WF
Persistence 12.2 12.2
Transparency 16.2 8.1
Flexibility 4.9 19.4
Maintainability 4.9 4.9
Extensibility 1.4 4.3
Risk 4.3 1.4
Cost 1.0 4.8
Total 44.8 55.2
18
0
10
20
30
40
50
60W
eigh
ted
Scor
e
BizTalk WFAlternatives
Multi-Criteria Decision Analysis
CostRiskExtensibilityMaintainabilityFlexibilityTransparencyPersistence
Figure 4.1: Graphical View of Results
The results show that BizTalk’s BAM Portal and Alert Manager resulted in a
significant advantage with respect to transparency, while its lower degree of risk as a
technology investment gave it an additional, minor advantage. Conversely, WF’s support
for state machine workflows and runtime modification resulted in a considerable
advantage with respect to flexibility, while its superior extensibility and substantially
lower cost gave it additional, minor advantages. Overall, the AHP calculations indicate
that WF is the superior solution, with a numerical advantage of just over ten percentage
points. It is also interesting to note that WF has a higher weighted score even when the
cost component is ignored.
19
5 Conclusions
The results of the evaluation indicate that BizTalk and WF are very similar in
some aspects and very different in others. The large difference visible in some factors is
mainly a natural result of the history and design philosophies of the two alternatives.
WF’s state persistence mechanism and business rules framework are obvious
carryovers from BizTalk. With respect to persistence and maintainability, both
alternatives provide a robust solution, and they fare equally well.
Whereas WF is a freely distributable developer framework, BizTalk is a robust,
enterprise-class server application, and this distinction is evident in the tracking tools
provided. In addition to providing the base tracking service, BAM provides several
delivery channels for the information, including the BAM Portal and the Alert Manager.
The most influential factor on the outcome was flexibility for human-driven
business processes. BizTalk was intended for EAI and B2B workflow, and this is evident
in its lack of support for state machine workflows or runtime modification, both of which
are supported by WF.
Ultimately, WF’s superior flexibility for human workflow makes it the more
appropriate choice for the AFE system. It is also important to recognize that even with
cost ignored, WF is still the better solution.
20
6 Recommendations
In light of the results of the analysis, this report recommends that the software
architect design a solution built on WF. The increased flexibility of the framework
makes it the more appropriate choice for a human workflow solution such as the AFE
system. Members of the development team should undergo whatever training is
necessary to familiarize themselves with WF. Previous experience with BizTalk may
also serve as an asset, as some of the concepts in WF are carryovers from BizTalk.
To compensate for the lack of some of the tracking tools that would otherwise be
provided by BizTalk’s BAM, the development team should provide transparency into the
workflow by leveraging a reporting framework such as SQL Server Reporting Services
and by creating a custom notification service. Neither of these is an exceptionally
complex task relative to the project as a whole.
There is a slightly higher risk associated with choosing WF rather than BizTalk,
mainly due to the possibility of breaking API changes and the need to eventually migrate
to the final release. The project manager should take these factors into consideration
when quoting time estimates and formulating a deployment plan.
21
References
[1] D. Hasselback, “Client Company’s quarterly profit”, The Financial Post, Aug. x,
2006, [Online], Available: http://www.canada.com/nationalpost/
financialpost/story.html?id=e250af6e-b7c8-471c-819b-48742743411e&k=69095,
Accessed: Sept. 5, 2006.
[2] J. Dodgeson et al., Multi-Criteria Analysis Manual, tech. report, Dept.
Communities and Local Govt., UK Govt., 2000.
[3] J. McCaffrey, “Test Run: The Analytic Hierarchy Process”, MSDN Magazine,
June 2005, pp. 100-107.
[4] “Enabling People-Ready Processes through Business Process Management
(BPM)”, white paper, Microsoft Corp., June 2006.
[5] “Business Activity Monitoring”, white paper, Microsoft Corp., April 2005.
22
Acknowledgements
I would like to thank Dr. Andrew Morton and Santheeban Kandasamy for
suggesting the use of MCDA for this report. They provided me with the background
necessary to perform my own research on MCDA.
I would also like to thank my colleague, Colleague, for his insight into the
application of BizTalk Server to human workflow systems. Mr. Colleague’s expertise on
BizTalk and his experience building similar systems proved invaluable.
23
Appendix A: The Analytic Hierarchy Process
The Analytic Hierarchy Process (AHP) provides a robust, quantitative analysis of
alternatives when dealing with multiple, subjective criteria. It converts qualitative
comparisons into weighted scores, which represent each alternative’s total value using a
single, useful metric. AHP is based on binary comparisons (that is, comparisons between
two items at a time), since it is easier to compare a pair of items than to rank a large set of
items.
The first step in AHP is to define a rating scale that converts qualitative
comparisons into numerical values that can be used in calculations. The analyst can use a
similar rating scale to evaluate both the importance of the criteria and the quality of the
various alternatives. Table A.1 illustrates a typical rating scale:
Table A.1: Rating Scale
Factor Subjective Rating
1 Equally important/Equally preferable
2 Slightly more important/Slightly preferable
3 Significantly more important/Significantly preferable
4 Much more important/Very preferable
5 Extremely more important/Extremely preferable
The next step is to assess the relative importance of the criteria, with the ultimate
goal of computing a set of criteria weightings. The analyst prepares a matrix with the
criteria listed along both the vertical and horizontal axes, as shown in Table A.2. Each
cell then represents the importance of the criterion on the vertical axis relative to that of
the criterion on the horizontal axis, in accordance with the rating scale. A criterion is
obviously equally important as itself, so the cells along the diagonal contain unity. A
value less than unity indicates an inverse relationship (i.e., the criterion on the vertical
axis is less important than the one on the horizontal axis).
24
Table A.2: Criteria Matrix
Pers
isten
ce
Tran
spar
ency
Flexib
ility
Mai
ntai
nabi
lity Ex
tens
ibilit
y
Risk
Cos
t
Persistence 1 1 1 3 4 4 4 Transparency 1 1 1 3 4 4 4 Flexibility 1 1 1 3 4 4 4 Maintainability 1/3 1/3 1/3 1 2 2 2 Extensibility 1/4 1/4 1/4 1/2 1 1 1 Risk 1/4 1/4 1/4 1/2 1 1 1 Cost 1/4 1/4 1/4 1/2 1 1 1
The analyst then computes the sum of each column, and divides each cell in the column
by that sum, yielding a normalized matrix, such as the one shown in Table A.3. To
account for any inconsistencies, the analyst computes the average of each row of the
normalized matrix; each row’s average represents the percentage weighting for the
criterion associated with that row.
Table A.3: Normalized Criteria Matrix
Pers
isten
ce
Tran
spar
ency
Flexib
ility
Mai
ntai
nabi
lity Ex
tens
ibilit
y
Risk
Cos
t
AV
ERA
GE
Persistence 0.244 0.244 0.244 0.260 0.235 0.235 0.235 0.243 Transparency 0.244 0.244 0.244 0.260 0.235 0.235 0.235 0.243 Flexibility 0.244 0.244 0.244 0.260 0.235 0.235 0.235 0.243 Maintainability 0.0816 0.0816 0.0816 0.0869 0.117 0.117 0.117 0.0978 Extensibility 0.0612 0.0612 0.0612 0.0434 0.0588 0.0588 0.0588 0.0576 Risk 0.0612 0.0612 0.0612 0.0434 0.0588 0.0588 0.0588 0.0576 Cost 0.0612 0.0612 0.0612 0.0434 0.0588 0.0588 0.0588 0.0576
The analyst uses a similar process to perform the actual evaluation. For each
criterion, she creates a comparison matrix to compare the alternatives against that
criterion. The analyst then normalizes the matrix and computes the average of each row,
which in this case represents each alternative’s score for that criterion. To obtain the
weighted scores, she then multiplies each alternative’s score by the criterion’s weighting.
25
Tables A.4 and A.5 show comparison matrices for the ‘flexibility’ criterion in initial and
normalized form, respectively.
Table A.4: Comparison Matrix
BizTalk WF
BizTalk 1 1/4
WF 4 1
Table A.5: Normalized Comparison Matrix
BizTalk WF AVERAGE WEIGHTED
SCORE
BizTalk 0.2 0.2 0.2 0.049
WF 0.8 0.8 0.8 0.194
After computing the weighted scores for all of the criteria, the analyst computes
the sum of the weighted scores for each alternative, yielding a set of total scores. The
analyst often multiplies these scores by a scale factor; for example, to use a 100-point
scale, she multiplies the values by 100. Table A.6 illustrates the computation of the final
scores.
Table A.6: Final Scores
Criterion BizTalk WF
Persistence 0.122 0.122
Transparency 0.162 0.081
Flexibility 0.049 0.194
Maintainability 0.049 0.049
Extensibility 0.014 0.043
Risk 0.043 0.014
Cost 0.010 0.048
Total × 100 44.8 55.2