Studying the evolution of skill profiles in distributed, specialization driven service delivery systems through work orchestration

Shivali Agarwal, Sreyash Kenkre, Vinayaka Pandit, Bikram Sengupta IBM India Research Laboratory

Bengaluru, India e-mail: {shivaaga, srkenkre, pvinayak, bsengupt}@in.ibm.com

Abstract—Many sectors of Services Industry like Software Services, Healthcare Services, Customer Care Services etc. have benefited from IT enabled, globally distributed service delivery mechanisms. The first generation delivery models were motivated mainly by cost arbitrage and featured dedicated teams for each major customer (or source of demand). We are now witnessing the emergence of delivery models that go beyond the cost arbitrage considerations. One such trend is the delivery of services via skill-specific service delivery centers. The centers are organized around the core skills of their employees. Instead of having dedicated teams, service requests are dynamically routed to the most appropriate delivery center based on required skills and availability. When executed effectively, it allows organizations to improve utilization measures and exploit locally available skills for global delivery. We consider a model in which a centralized planner orchestrates a network of skills-driven delivery centers to serve a global pool of requests. The delivery centers are semi-autonomous in the sense that they can pursue certain local objectives like diversifying into new skills or improving skill levels as long as they do not significantly hamper overall service delivery. In this paper, we study evolution of skill profiles of the delivery centers from global and local perspectives under this orchestration model.

In our model, a stream of service requests, tagged by required skill type and a measure of workload is received by the central planner. The planner orchestrates the delivery by assigning the requests to the centers while matching the skill type and capacity. Ideally, the planner would like to assign the request to optimize some global objectives. However, organizational dynamics require that the centers be allowed to pursue local objectives without significantly affecting overall delivery. To achieve this, we propose the following mechanisms of assigning requests. In the "push" mode, the planner assigns the requests so as to optimize global objectives. In the "pull" mode, it allows the centers to compete for a subset of requests to pursue local objectives. Specifically, the planner auctions a subset of requests and the centers compete for the auctioned requests if they are aligned with local objectives. To enable the auction mechanism, the centers are assigned imaginary budgets. They have to utilize the budget to pursue local objectives. We anticipate scenarios when centers require assistance in pursuing their local objectives. The "aided push" mode allows the planner to aid a center in taking on requests which it would not be able to execute without the aid.

We have built a simulation framework aimed at studying the evolution of the system under different scenarios, characterized by: number of delivery centers, their sizes, requests arrival patterns, initial skill profiles, different local objectives, and different global objective of the planner. Our simulation framework and the experiments are designed to obtain important insights on the design of specialization driven service delivery systems through work orchestration. We present extensive experimental results which study various aspects of such a delivery model and present conclusions in the form of design guidelines.

Keywords- factory model, service delivery, work allocation, auction, push based allocation, pull based allocation, simulation

I. INTRODUCTION

A. IT Enabled Globally Distributed Service Delivery Information Technology has enabled remote and globally distributed delivery of services and has thus, transformed many industrial sectors. This trend, which began by first transforming the mechanisms for Software Services delivery, has broadened its scope to other sectors amiable to the use of IT: Healthcare Services, Customer Care Services, Bio-tech services etc. For example, a significant portion of medical examination reports are routinely processed from global delivery centers in India. A notable example of globally distributed delivery of services in a traditional industrial setting is the orchestration of a supply chain for apparels by Li&Fung [9]. The abstract model for distributed service delivery we consider is the following. There is a particular service that an organization is offering. The clients of the service generate a sequence of specific service requests, all within the purview of the service being offered. Delivering the service means servicing the requests in a timely and competent manner. The delivery model that we are interested in is a globally distributed network of delivery centers, which are engaged in delivering the service by resolving the stream of service requests in a client-oblivious manner. For example, in the context of Software Services, an organization supporting a product receives a stream of modification or enhancement requests and delivers the service by carrying out the modifications and enhancements using a network of globally distributed offshore development centers. Typically, in order to deliver services,

2011 Annual SRII Global Conference

978-0-7695-4371-0/11 $26.00 © 2011 IEEE

DOI 10.1109/SRII.2011.32

215

2011 Annual SRII Global Conference

978-0-7695-4371-0/11 $26.00 © 2011 IEEE

DOI 10.1109/SRII.2011.32

215

2011 Annual SRII Global Conference

978-0-7695-4371-0/11 $26.00 © 2011 IEEE

DOI 10.1109/SRII.2011.32

215

2011 Annual SRII Global Conference

978-0-7695-4371-0/11 $26.00 © 2011 IEEE

DOI 10.1109/SRII.2011.32

201

2011 Annual SRII Global Conference

978-0-7695-4371-0/11 $26.00 © 2011 IEEE

DOI 10.1109/SRII.2011.32

201

the organization needs to handle tasks requiring multiple skills. For example, to deliver software services in the area of operating systems, the different areas in which we need skilled people are I/O Subsystem, OS Internals, Device Drivers, Networking Subsystem, Graphical User Interface etc.

B. Previous Generation Delivery Models The previous generation of service delivery was mainly motivated by cost considerations. The delivery centers were located in low-cost geographies that had the required access to skills (or could achieve minimum required level via training) [10]. A typical service delivery organization served multiple clients by setting up separate, dedicated teams for each client. This model suited the clients particularly well as they could hold the dedicated teams responsible for their performance. But, at a high level, it had two disadvantages for the delivery organization. First, the capacity had to be built to handle near-peak requirements and hence, deal with the potential problem of underutilization. Secondly, the dedicated teams had to have all the multiple skills required to deliver the service. For example, for a company wanting to deliver software services for an Operating System, this model was not suitable if the ideal access to skills in I/O Subsystem was in Geography A and ideal access to skills in OS internals was in Geography B. In the past few years, the service delivery organizations have been experimenting with different models to deal with these intertwined problems of, (i) capacity management and (ii) exploiting the locally available skills in a globally distributed delivery set-up. The service delivery industry has tried to deal with the twin problems of capacity management and access to skills in a somewhat decoupled manner. From a capacity management perspective, most of the approaches have been variants of the so-called “minimum commitment” model. Roughly, the service seeker commits to a certain minimum volume of demand, at different granularities of time, which in turn helps the service provider to carry out capacity planning. This model turns out to be a major limitation for the customers especially when sourcing from multiple service providers. As the service quality and responsiveness of the providers start varying, the customers start feeling that the minimum commitment clause acts as a tie-in mechanism to source from an inferior service provider. Increasingly customers are trying to avoid such models. On the skills front, the organizations have mainly used “training” as the mechanism to deal with access to skills, and recover the training costs by (i) charging the customers and (ii) creating disincentives for trained employees to leave the organization. Not surprisingly, these quick-fixes have turned out to be unsatisfactory.

C. Emerging Factory Models Consider a service organization that has a set of clients (client here refers to an organization that is being served) and offers a service requiring a pre-determined set of skills for successful delivery. The variability of demand generated by individual clients is difficult to deal with as it is critically dependent on its internal business situations, which is the client’s private information. On the other hand, it is relatively easier to deal with variability in the total demand generated by the entire set of clients. It can be guided by aggregate market research, industry news etc. This in turn means that it is easier to deal with the variability in demands for individual skills when aggregated across all the clients. So, instead of having dedicated teams, organizing the service delivery in terms of delivery centers that specialize in specific skills has the following advantages: (i) better handling of capacity management issues (ii) exploit local availability of skills in a remote, distributed service delivery network, and (iii) achieve improved productivity and better quality of service due to specialization. At a high-level, the “factory model”, emerging as a popular service delivery model, is motivated by the above factors. It aims to achieve the advantages of the well-known factory model in traditional industrial manufacturing settings. The main idea is to organize teams based on their skill competency (rather than having dedicated teams for individual clients), and the service requests (across all clients) are routed to the appropriate teams depending on the skills required. In this paper, we consider different aspects of work allocation in specialization driven factory model for service delivery. In our model, there is a set of remote delivery centers called Specialized Delivery Centers (SDC) each one of which is divided into a set of teams each of which specializes in a specific skill. The client requests, after being appropriately decomposed into atomic units (called work packets), each requiring a specific skill, are routed to one of the teams in a SDC that possesses that required skill. In our setting, the SDCs have some limited autonomy to pursue some local objectives, while there is an overarching strategic planner which aims to meet some organization-wide global objectives. The interplay between local and global objectives, and the different work allocation strategies to meet these objectives, form an important part of our study. We build a simulation framework to study the evolution of such a delivery system from the perspective of skill profiles of the SDCs and the meeting of local/global objectives, and present some experimental results that give guidelines for some typical design questions for our factory model.

II. MODEL

A. Our Model Our model for service delivery is based on the SDCs. As highlighted in the introduction, the SDCs offer several

216216216202202

advantages, especially in dealing with the problems of capacity and skill management faced in the first generation service delivery systems. We model the service delivery organization as a geographically distributed network of SDCs. Let us say that S denotes the set of skill types that the delivery organization needs to deliver the service. Each SDC is specified by the set of skill types that it possesses and the capacity that it has for each of them. To simplify our experimental setup, we capture the capacity of an SDC for a skill type in terms of the number of people possessing that skill type. For each person, a uniform work-calendar of K hours-per-day is assumed. As mentioned in the introduction several considerations specific to the local geography, like access to skills and cost structure determine the establishment of SDCs. The overall delivery of the services across the different SDCs is orchestrated by a centralized strategic planner (SP). The SP receives a stream of atomic work packets, each specified by the tuple (Skill Type, Demand Volume, Duration). The SP is responsible for allocating the incoming stream of requests to the appropriate SDCs for delivery. The figure below (Figure 1. shows the high-level framework of our service delivery. In an ideal setup, one would like to provide complete control over the assignment of requests and schedule of delivery to the SP. But in reality, the SDCs are motivated by their ambitions of growth along different dimensions and are constrained by their local settings and competition. Therefore, an important question mark over the factory model is: “can the factory model work effectively when the SP is required to grant limited form of autonomy to the SDCs?” The goal of our work is to seek answer to this question via the route of a simulation platform for the factory model. As the system is orchestrated by the SP via a network of SDCs with limited autonomy, we need a mechanism to capture the interplay between the objectives of the SP and the limited autonomy of SDCs. We do this via the notions of global objectives and local objectives. The SP pursues global objectives which capture the desirable global properties the system should possess such as efficiency, load balancing, fault tolerance, risk minimization etc. The SDCs pursue local objectives. The local objectives of an SDC are driven primarily by (i) access to skills in the local geography and (ii) business strategy such as high volume throughput or skill diversification or specialization etc. Our model allows the SP and SDCs to pursue their objectives via different work allocation mechanisms. The SP uses the Push mode of allocation to directly assign work packets to the SDCs in a manner that helps it to achieve its global objectives. When it pushes a work packet to an SDC, the SP makes sure that there is sufficient capacity in the required skill inside SDC. The SDCs cannot reject work packets assigned to them in the push mode. To allow the

SDCs to pursue their local objectives, the model allows them to seek desirable work packets via Pull mode. In the pull mode, the SP makes public a work packet that is up for assignment, and the SDCs compete among themselves to pull it. Figure 1. High level framework of our delivery model

The goal of the SP in pull mode is to allow the SDCs to pursue their individual local objectives; i.e. the pull is a mechanism for social welfare. Therefore the SP needs to assign the work packet to the SDC that has the highest utility for it. Since the utility of a work packet for an SDC depends on the instantaneous state and goals of the SDC, the SP has no efficient means of calculating this. So the SP needs query each of the SDCs and get a correct value of their utilities for the work packet. Conducting an auction is a very flexible and efficient way of doing this [14] . The auction can be a very specific customized mechanism, or a standard VCG auction. We use the latter. For this, at certain time intervals, the SP reviews the SDCs (their size, skill type distribution, local objectives etc.) and allocates budgets to them. The SDC’s then use these budgets for bidding for work packets in the auction. For each auction, the SP collects these bids from those SDCs that have the capacity to process the work packet, and depending on the auction mechanism, chooses a winner and the price at which the work packet is given to the winner. Once assigned, the winner has to process the work packet. The use of auctions in software service delivery has been explored before (see

Assignment of work via Push or Pull

Stream of Work packets tagged by Skill Type

Centralized Strategic Planner (SP)

SDCs: Specified by Vector of (SkillType, Capcity).

SDC SDC SDC

217217217203203

[15] and its references). However, all these have been attempts at a pure pull mode and do not explore the concepts of trade off between global and local objectives. We measure the success of the SP and SDCs in pursing their objectives in terms of how close they get to their objectives in terms of volume and desired skill profile. The main question concerning the factory model is “Can both the SP and SDCs pursue global and local objectives without affecting the overall service delivery?” If the answer to the first question is “yes”, it would imply a significant vote of confidence for the factory model. Our simulation framework is one way to obtain answers for these questions.

B. Related Work There is a significant body of research in the broad area of organizational design that is relevant (and mostly complementary) to our work. In particular, the use of model-based simulation as an aid to the development of complex organizational theory – that is often difficult to formulate in closed-form mathematical terms – has been tried out for several years in areas such as organizational choice [1], organizational learning [2] etc. More recently, organizational behaviors and outcomes are being increasingly recognized as the result of the complex interactions of multiple interdependent processes, and understanding the resultant nonlinear system behavior is often best approached through modeling and simulation [3]. For example, [4] uses computational modeling to help study the fundamental relationships among structure, performance and environmental dynamism, the basic argument being that organizations with too much structure tend to be inflexible, while those with too little structure tend to be inefficient. The amount of structure is formalized in terms of the number of rule-based constrained actions vis a vis actions that are improvised to adapt to available opportunities, with performance being measured in terms of payoffs associated with the opportunities that the organization is able to exploit in this way. While we also explore the trade-offs between efficiency and flexibility in an organization, we do so in the context of different levels of decentralized decision-making, and thus our overall focus is on the interplay between local vs. global organizational objectives and semi-autonomous work allocation, unlike [4]. Closer to our work, [5] uses computer simulation to model a set of firms competing in markets for opportunities to invest in existing capabilities and acquire new ones. The authors study the conditions under which there are performance benefits from possessing a superior ability to search for new capabilities, taking the initial capability endowments into consideration. This has some similarities with our study of how different service delivery centers of an organization can compete for work and how they can acquire new skills and diversify. However, in our model, the centers are not fully autonomous entities, and there is an overarching organization and a global set of objectives that constrain the

actions of the different centers. Also, our model of competition, based on a level playing field where each eligible firm can bid for a new work-stream is very different from [5], where firms are authorized in random order to take up a work-stream of choice from the set of currently available opportunities. In [6], the authors develop a model of hierarchies within an organization based on the allocation of authority. They analyze the optimal degree of decentralization based on the view that some entities are engaged in coordination over a large set of assets, while others are engaged in specialization of limited set of assets, and suggest that individuals with a broad remit i.e. whose tasks cover a large subset of assets, should appear higher in the chain of command than those with narrow remit. Such a design is inherent in our model of the strategic planner that coordinates work allocation globally across the different delivery centers specializing in different skills, while controlling the level of decentralization through a mix of push and pull. If we look beyond the literature on organizational theory and management, there has been significant work in the general area of scheduling/resource allocation, which is a fundamental problem that has been very well-studied over the decades [11], [12], [13]. Typically, solutions to such problems involve centralized decision-making to determine which job should be allocated to which entity (machine, agent etc), so that certain operational objectives (e.g. completion time, makespan etc) are optimized. In our setting, the resource allocation objectives are more strategic in nature, given our focus on longer-term skill acquisition and diversification within an organization. Also, we explore de-centralized decision-making, whereby service delivery centers are provided some (limited) flexibility to try and obtain work that will help promote their local aspirations and objectives. There is growing appreciation of the need to support such empowerment in human and organizational endeavor, and heterogeneous work allocation models that combine pull-based and push-based job allocation strategies in the services industry have been studied in domains such as telecommunication field workforce management [7], and health-care [8].

III. SIMULATION FRAMEWORK

A. Simulation Overview Our platform is developed as a discrete event simulation system. The main events which trigger the actions of the delivery system are the arrivals of the atomic work packets. Each work packet is specified by (Skill Type, Demand Volume, Duration). The SP pursues a global objective which is specified by different combination of weights on functions like throughput, load balancing, optimal utilization etc. The SDCs pursue local objectives like throughput, skill diversification, capacity enhancement etc. The SP applies a decision procedure for each work packet

218218218204204

and depending on its attributes determines whether to assign the work in the push mode or pull mode. If it is the push mode, then, the SP determines the SDC which is best suited from the point of view of making progress on the global objective. If it is the pull mode, then, the SP simply conducts an auction by announcing the attributes of the work packet that is being auctioned. The individual SDCs assess the auctioned work packet from the point of view how much it can help them progress on their local objectives and put a bid which is proportional to the extent of progress. The SP assigns the work packet to the highest bidder. In the push mode, a work packet gets dropped if there is no capacity in any of the SDCs that matches its demand volume. In the pull mode, a work packet gets dropped if none of the SDCs bid for it. We also study a pull-push mode in which if the pull fails, then, SP does the assignment using push mode. When a work packet is assigned and when it is completed, the assigned SDC does the necessary book-keeping to update its available capacity and calendar information.

Figure 2. Service Delivery Simulation Framework

B. Components Of The Simulation Framework We now describe the important components of our simulation platform.

1) Workload Generator

The workload generator is the module which generates the stream of work packets that the delivery system handles. Currently, it generates a stream according to a pattern commonly observed in the request arrival patterns across multiple service delivery projects within IBM. The request arrival pattern for each skill type follows a Poisson distribution. The stream is an interleaving of the streams for all the skill types in a time-sorted manner. The volume and duration of the work packets are generated in a pre-specified range uniformly at random. But, depending on the domain, they can be made to fall into well known distributions such as “power law” distributions. We also generate some special streams designed to answer some of the questions raised in Section II.

2) Work Packet Assignment Manager Once a packet arrives in the system, it passes through several stages before exiting the system. The first stage decides the mode in which the packet will be processed. In the pure push or pure pull mode this, there is no decision to make in this stage. In the combination mode, the assignment manager has to first decide whether it will be pushed or pulled. To enable the SDCs to make progress on local objectives, there needs to be sufficient number of packets that the SDCs can compete for. So the assignment manager sets a policy which ensures this. One simple policy which we explore is to first try to assign the work packet in the pull mode, and if no SDC bids for it, to assign it in the push mode. Another policy is to allocate a fixed fraction of the total number of work packets in the pull mode, and the rest in the push mode. Suppose the fraction is f, then, the assignment manager puts every (1/f)th packet via the pull mode. To do so, it conducts an auction by specifying the attributes of the packets. If the auction fails to generate any bid from the SDCs, then, if there is spare capacity in the required skill type in any of the SDC, then, the SP assigns the packet to the best suited SDC via the push mode. Whenever the SP assigns packets in the push mode, it determines the assignment that makes the best progress on the global objectives.

3) Objectives and Policy Maintenance Module One of the most important aspects of the delivery model we study is the set of local objectives of the SDCs and the global objective of the SP. We also have certain policies that the organization follows while trying to achieve these objectives. The SP has objectives that consider the system as a whole. They are (i) growth of SDCs (ii) skill type diversification at specific SDCs to increase fault tolerance, referred to as skill redundancy, and (iii) load balancing across SDCs and throughput maximization. The SP specifies its overall objective by specifying different weights to these objectives. For example, if all are equally important, they will have a value of 1/3.

Directly Assign Put a bid

Workload generator: generate a stream of work packets.

Work Packet Assignment Manager

Objective and Policy Maintenance Module

Auction Manager: Conduct an auction and assign it to winner

SP

Pull Mode Push Mode

SDC

Capacity & Policy Manager

Capacity & Policy Manager

SDC

219219219205205

During the delivery process, the SDCs accumulate credit for the work packets that they process. This credit is measured in terms of the hours that the SDC has worked in a particular skill type, and the volume of that skill type it processed. At every pre-specified quantized level for each skill type, we make them eligible to exchange the credit for increase in its capacity for the corresponding skill type. Similarly, at every pre-specified quantized level aggregated across all skill types, we allow the SDC to exchange the credit for getting access to a skill type that it does not have in its portfolio. The local objectives for the SDCs are (i) increasing overall capacity (ii) increasing capacity of specific skill types (iii) obtaining skill types that it currently does not have. These objectives can be specified as follows. For overall capacity, it just specifies a multiplicative factor (>1) by which it aims to increase its overall capacity. For each existing skill type it specifies the goals similarly. For missing skill types, it expresses its aim of diversification by a Boolean value. The policy maintenance and application module does the following. It maintains the different quantized levels for each skill type at which accumulated credit can be exchanged for increases in capacity. It maintains the quantized levels for aggregate credit which can be used for diversification. These policies can be applied by both SP and the SDCs in a manner that allows them to make progress on their objectives. The policy manager specifies the order in which the SP and the SDCs get to apply them. After completing a packet that an SDC pulls, it gets the priority for applying the policies first. After a packet is completed in the push mode, the SP gets the first chance to apply the policies.

4) SDC Side Capacity and Policy Manager This module manages the capacity of the SDC, applies the local policies as required from the perspective of the local objectives, and responds to the auctions conducted by the work packet assignment module. The capacity is managed at the level of skill types. For each skill type, the SDC maintains “current capacity” and a calendar. The calendar essentially specifies “available capacity” at each point in time for a suitable window size. The window size can be anything larger than the largest packet duration that the delivery system handles. For a work packet to be scheduled in the SDC, from “current time” to “current time + duration of the packet”, the available capacity in the calendar must be greater than or equal to the packets’ demand volume. If it is indeed scheduled, then, that much capacity is subtracted from the available capacity at each of the time steps from “current time” to “current time + duration of the packet”. The SDC management is also responsible for two more functionalities. First, whenever it has accumulated enough credit to exercise local policies, it decides which policy to apply. This decision is made such that, as a result of the policy application, there is maximum reduction in distance to the desired local objective. Second functionality is that of

bidding for the packets put on auction. In this case, it does the following thought experiment: suppose the auctioned packet was to be allocated to itself, what is the reduction in its distance to the local objective? If its remaining budget is B, it makes a bid equal to = B * score, where “score” is a normalized number between 0 and 1 which indicates the extent of reduction in distance to the local objective.

5) Auction Manager The auction manager is a simple module which puts the packet for an auction by communicating its attributes to the SDCs. It receives the bids which are computed as described above. It then follows a simple VCG auction [14] model in which the bid with the highest amount is declared the winner and pays a cost computed according to the VCG auction mechanism.

6) Distance Computation For every objective in the system, we need a metric that gives us an idea of how close we are to achieving the objective. This distance function is used by the system components for the following functions.

1. When the SP first presents money to the SDCs for participating in the auction, it needs to get an idea of how much money an SDC will require. An SDC with ambitious objectives will need a larger quantum of money than a less ambitious SDC. For this, the SP uses the distance functions of the objectives of the SDC to compute the total distance the SDC needs to travel.

2. When an SDC participates in an auction, it needs to calculate its bid. For this it needs to get an idea of the extent to which its objectives will benefit if it is allocated the work packet. This is done by calculating the amount by which its distance to its objectives decreases if it processes the work packet.

3. When the SP pushes a work pipe, it needs an idea of how close it will be to its objectives by assigning the work packet to a particular SDC.

The objectives we consider are mainly of two types: growth and diversification. In each case, the objective has a numerical initial value, and a final value. For example, if an SDC has a capacity of 100 units for skill type 1, and it wants to increase it to 200 units, then the initial value is 100, and the final value is 200. Similarly, if the SDC wants to acquire a new skill type, then this objective has an initial value of 0 (skill type not present) and a final value of 1 (skill type present). Also, at any given point in time the current value of the desired objective can be calculated by looking at the SDC/SP. For example, if at time t, the above SDC has acquired a capacity of 150 units of the skill type 1, the current value of the objective is 150. We can now define the distance function as follows. Let x0 be the initial value of the objective, xt its current value, and xf be the desired final value. Then the distance (dist) is defined as follows:

220220220206206

10000⎟⎟⎠

⎞⎜⎜⎝

⎛−−=

xxfxxtdist

This can be interpreted as the percentage amount the SDC or SP has to travel to achieve the objective.

C. Discussion We would like to highlight some of novel aspects of our simulation framework. Firstly, in the context of the factory model of service delivery, it helps to study the realistic setting in which the SDCs pursue local objectives. Our representation of the local and global objectives, and our ways of computing the distance provides a formal framework in which the evolution of an organization can be studied and may have broader application in computational analysis of organizations. Finally, the use of budget allocation as a mechanism for pursing local objectives, and the auction framework for the SDCs to compete for packets takes the earlier work of use of auctions by individuals for gains which was typically based on a single factor.

IV. EXPERIMENTAL SECTION

A. Meta Questions The purpose of simulation based experiments is to address the following questions that are relevant to understand the evolution of the system and the behavior of different types of SDCs.

1) How does the work distribution strategy in terms of push and pull affect the system in reaching their desired state at the global and the local level? Further, what is the effect of alignment (or misalignment) of local and global objectives on the ability of the system to meet both local and global objectives? It would indeed be a strength of the factory model if the system can meet both global and local objectives even when they are not perfectly aligned

2) What role do evolution policies play in the evolution of the system, especially of different sized SDCs? Does the behaviour of the system change when different policies are applied for same work distribution strategy? If yes, how and why?

B. Design of Experiments The experiments are designed around the meta-questions discussed above. The experiments are carried out by the simulation of the factory model presented in detail in Section III. The configurable parameters of the simulation are discussed below.

1) Workload Configuration The workload consists of 6 skill types. Each work packet in the workload is classified as small, medium or large based on duration and required capacity. A small work packet is smaller in both duration and required capacity compared to medium and large while a large work packet is larger in

both duration and required capacity compared to others. The parameters in workload generator can be set to generate the desired number of work packets of desired skill and size.

2) System Configuration a) An SDC is allowed to have one of the three sizes of

capacities for each skill type – small, medium and large. The ratio that we have considered is 1:2:4. An SDC can have any subset of the 6 skills that we have generated the load for. Based on this, the notion of small, medium and large SDCs is defined as follows : A small SDC will have only the small sized capacity for all the skill types that it has while medium SDC will have only the medium sized capacities for all the skill types that it has and analogously for large SDC.

b) Policies specify, for each pair of (skill type, current capacity), the amount of work hours with that skill type required to achieve an increase in capacity expressed in terms of a fraction of the current capacity. Similarly, this is also specified for diversification purpose by specifying total amount of work hours required to invoke diversification policy. The policies are independent of the size of SDCs. The value of the large, medium and small capacity for a skill type and the number of small, medium and large SDCs in the system and the policies that are responsible for evolving the system define the system configuration.

3) Objectives and the Notion of conflicts in objectives A global objective is cumulative in nature. That is, a system may have a global objective that it wants to increase the capacity of a skill type from x to 2x where x is the cumulative capacity across SDCs. Some important points to remember while configuring objectives are: Global objectives are defined at SP level whereas local objective are defined only at SDC level. Global objectives are invisible to SDCs and no SDC is aware of the local objectives of other SDCs. A global objective can conflict with a local objective on two accounts: i) the values by which an SDC wants to grow in a particular skill type is much more than what the global objective for that same skill type is i.e. aligned v/s misaligned, ii) an SDC wants to grow in a skill type for which there is no growth specified by the SP in the global objectives or vice versa i.e. orthogonal objectives. Based on the above workload and system configurations, the experiments that were carried out are described below. Experiment I : Evaluate different work allocation strategies This experiment is designed to obtain insights on the first meta-question. In particular, we show that with the adoption of “pull+push” based allocation, the system evolves better as compared to pure “push” based allocations. For these experiments we consider three systems with different objectives and number of SDCs. These are representatives of the bigger systems and the results obtained by running simulations for these systems can be generalized for the

221221221207207

bigger systems. The three representative systems are described below. System 1: This has 6 SDCs: 1 large, 2 medium and 3 small. Each SDC has a mix of diversification and growth objectives such that some of the objectives are aligned with the SP, some are misaligned with the SP, and the rest are orthogonal to the SP objectives. System 2: The number and sizes of the SDCs in this system is the same as that in System 1; however, the objectives are different here. There are no diversification objectives in the system. All the growth objectives of the SDCs are orthogonal to the objectives of the SP. System 3: This has 10 SDCs: 3 large, 3 medium and 4 small. This also has a mix of objectives as in system 1 with the exception that two of the large SDCs do not have any local objectives. For each of the above systems, we conducted experiments with 2 different workloads. The work allocation strategies, also called modes of allocation, used were (i) Pure Push (ii) Pure Pull (iii) Pull+Push. The interpretation of pull+push mode is: SP puts up for auction every work packet it gets. If some SDCs bid for it, the SP passes it to the winner of the auction. However, if no SDC bids for it, the SP adopts the push mode for the work packet, and assigns it to a suitable SDC. For each of the runs of the simulation, we recorded the number of work packets that were pushed, pulled or were dropped from the system. We also have a parameter called as the Distance From Objectives, for the system, which we track with time. This parameter is the sum of the relative distances left to be covered to achieve the respective objectives summed over the objectives of the SP and all the SDCs in the system i.e. it is the cumulative distance to be covered by the system. We plot this function as a function of time. The plot of this function for System 1, for one of the work loads is as shown below (Figure 3. ). For each of the 6 experiments we observe patterns which are similar to the one shown in this graph. For each of the experiments, the graphs of the Distance From Objectives when plotted as a function of time for push, pull and pull+push combinations was observed to be very close to each other. The graph of the pure pull was usually higher than the other two graphs as it did not take into account the SP objectives. Since the pure push can process more packets than the pure pull case, it performs better than the pure pull. The pull+push mode, however, seems to perform equally well, and sometimes even outperforms pure push. This observation on the evolution of the system from the combined viewpoint of global and local objective is indicative of the inherent strength of the factory model.

Figure 3. Distance From Objectives for System 1

We also compare the number of work packets that the system is able to process in each of the modes. In a pure pull mode, the SDCs will only bid if the work packet helps it achieve some objective. Thus, all packets that do not help any SDC will be dropped. However, in a pure push mode, the SP needs to ensure that the system does not drop too many packets. Thus it always tries to assign a packet to some SDC irrespective of whether that work packets helps the SP make progress towards achieving some global objective. We see below in TABLE I. , that this is indeed the case. Note that the pull+push mode is almost as good as pure push, and in some cases better. The first column gives the number of work packets the system pushes to the SDCs, the second gives the number of work packets that are successfully auctioned by the system. The last column gives the number of work packets that the system is able to process successfully. We note that the pull+push mode is not a simple sum of the packets of the pull and push mode. This can be seen by noting that when a work packet is allocated by the pull mode, a certain capacity is used for processing it. The capacity is not available for the push mode. Thus the pull and push interact in the pull+push mode, to give a different distribution of work loads than the individual modes. In some cases it does this constructively enough to be able to process more work packets than the individual modes.

TABLE I. NUMBER OF WORK PACKETS PROCESSED

Pushed Pulled Processed Simulaiton1 Push 4524 0 4524 Pull 0 2147 2147 Pull+Push 2425 2122 4547 Simulation2 Push 2749 0 2749 Pull 0 827 827 Pull+Push 1962 780 2742 Simulation3 Push 4226 0 4226

222222222208208

Pull 0 1189 1189 Pull+Push 3085 1156 4241 Simulation4 Push 2290 0 2290 Pull 0 433 433 Pull+Push 1989 367 2356 Simulation5 Push 3896 0 3896 Pull 0 1716 1716 Pull+Push 2254 1688 3942 Simulation6 Push 2779 0 2779 Pull 0 807 807 Pull+Push 2100 744 2844

Experiment II: Evaluate the role of push/pull/pull+push mode in meeting the objectives of the SP and SDCs. In this experiment, we again consider the first meta-question from a slightly different point of view. The simulations used for this experiment are the same as the ones used in Experiment I. We now study the trade offs of the modes push/pull/pull+push in achieving the objectives. For a given system, we would ideally want the SP as well as the SDCs to achieve their goals. However, due to various conditions like competition from other SDCs in the auction, load balancing by the SP, lack of particular skill types in the work packets etc., some of the objectives may not be reached. So to evaluate how the system has performed we use three parameters: (i) distance of SP from the global objectives (ii) number of SDCs that have achieved their goals (iii) the maximum of the cumulative distance to be covered by each of the SDCs, called max-local-distance. The parameter max-local-distance above is calculated as follows. Suppose S(1), S(2),…,S(k) are the SDCs. Let SDC S(i) have the objectives O(i,1), O(i,2),…, O(i,n). Suppose at the end of the simulation, objective O(i,j) needs to cover a relative distance of d(i,j). Then the cumulative distance that S(i) has to cover is the sum over all d(i,j), as j varies from 1 to n. Let this be denoted by D(i). Then, the maximum over all D(i) as i varies from 1 to k, is taken to be parameter max-local-distance. This parameter essentially gives the largest dissatisfaction any SDC can have (see TABLE II. ). We would ideally like this parameter to be small, to avoid the extreme unbalanced cases where some SDCs have achieved their objectives, while some SDCs are far from their objectives. In each of the above simulations we observed that the dissatisfaction in pull+push mode was the least, while it was maximum in the pure pull mode, especially when the objectives of the SDC were orthogonal to those of the TAC.

TABLE II. MAXIMUM DISSATISFACTION

SP initial distance

SP final distance

Max SDC initial cumul distance

Max SDC final cumul distance

SDC dissatisfied

Sim1 Push 5100 0 400 127 2/6 Pull 5100 99 400 100 3/6 Pull+Push 5100 0 400 100 3/6 Sim2 Push 5100 0 400 100 1/6 Pull 5100 99 400 100 1/6 Pull+Push 5100 0 400 100 1/6 Sim3 Push 2700 0 100 3 1/6 Pull 2700 300 100 8 1/6 Pull+Push 2700 0 100 0 0/6 Sim4 Push 2700 0 100 0 0/6 Pull 2700 300 100 0 0/6 Pull+Push 2700 0 100 0 0/6 Sim5 Push 6300 0 400 119 3/6 Pull 6300 48 400 100 3/6 Pull+Push 6300 0 400 100 3/6 Sim6 Push 6300 0 400 119 2/6 Pull 6300 48 400 100 1/6 Pull+Push 6300 0 400 100 1/6

We also conducted experiments to study the effect of a very large (giant) SDC on the system. We simulated the above systems for each of the three modes and introduced a very large SDC (i.e. 3 to 4 times the total capacity of the usual large SDCs). In the pull mode the effect of the giant is seen only when its objectives are extremely ambitious. These are similar to the effects when normal SDCs have an extreme objective. For the push mode, the allocation of work packets takes place oblivious to the local objectives. However, the SP is able to reach its objectives (scaled for taking into account the large SDC) faster as there is a large increase in capacity of the system. The case of the pull+push mode is similar to the push mode. Here, when the SDC has no objectives or has objectives orthogonal to that of the SP, the rate at which the SP achieves its objectives is slower than when the objectives are aligned with that of the SP. The figure below (Figure 4. ) shows the case when System 1 above was simulated with the giant SDC. We have plotted the distance of SP from its objectives for each of the three modes pull/push/pull+push, as a function of time. We see that the time taken by the SP to reach its objectives is about 50 time units earlier than the others. This suggests that the system functions efficiently when the SP objectives are aligned with the objectives of the large SDCs. However, note that to achieve the above difference in time, we needed an SDC which was almost 4 times the size of the usual large SDC. Thus, there is a built in robustness in the system. This comes from the policies, which demand a larger number of work hours from larger SDCs for the same percentage of

223223223209209

growth or diversifications. This also implies that unless there is an abnormally large SDC, our system allows the SDCs and the SP to pursue mutually orthogonal goals without hampering its performance and evolution.

Figure 4. Pull+Push Mode With Giant SDC

Experiment III: Evaluate the role of budget allocation on the pull mode, and the concept of aided push This experiment is designed to obtain insights on the second meta-question. At the start of the simulation, the SP allocates budgets to the SDCs with which they can participate in the auctions. This initial budget allocation can depend on various factors like the size of the SDC, the distance it has to travel to meet its objectives etc. Note that we put in the constraint that, while allocating the initial budgets, the SP cannot give preference to SDC’s with local objectives aligned with the global objectives, as this will discriminate against SDC’s with misaligned or orthogonal objectives. We used the above systems for two types of budget allocations: (i) based on size of the SDCs and (ii) based on the distance they need to travel to meet their local objectives. Of these, the second policy was found to be the best, while the first was oblivious to the local objectives. We also explored if the SP can allocate some extra budgets to SDC’s that could not meet their local objectives, as they repeatedly lost in auctions. We call the provision of such an aid to an SDC, as an aided push. For this we find that aided push has to be used judiciously. Continuing to aid a weak SDC can harm other SDCs. We show the results for a system of 5 SDCs in the pure pull mode in TABLE III. below. In the experiment, we simulated the pure pull mode, with the initial budget allocation based on the distance the SDCs have to travel to meet their objectives. We targeted SDC 1 for the aided push, and increased its initial money allocation. In the table below left hand column gives the SDC being referred to. Each of the columns is labeled by the factor by which we have increased the initial budget for

SDC 1. The entry in the table gives the final distance to local objectives of each of the SDCs, at the end of the simulation.

TABLE III. EFFECT OF AIDED PUSH TO SDC1

Factor x 1 Factor x 2 Factor x 5 Factor x 6 SDC 1 144 124 14 0 SDC 2 147 148 151 152 SDC 3 50 52 78 82 SDC 4 41 59 77 81 SDC 5 61 63 81 84

We see that while increasing the initial budget allocation benefits SDC 1, the final distance to objectives of the other SDCs keeps increasing. Thus the overall sum of the distance to objectives across all SDC’s also increases, thereby harming the social welfare aim of the auction. Thus, such an aided push has to be used prudently, so as not to harm the system. Experiment IV: Evaluate the role of policies in meeting the local growth objective for a given work distribution strategy This experiment addresses the second meta-question. The motivation of this experiment was to determine the factors that affect the growth of the system when the system is evolving under the control of only global objectives (push mode) versus only local objectives (pull mode). The various parameters that will be varied to study the role of policies in the system evolution in this experiment are:

i) Workload composition in terms of the numbers of work packets of different sizes.

ii) System composition in terms of the numbers of different sized SDCs.

iii) Global and local objectives in terms of conflicting (misaligned) or aligned. A conflict is in terms of the scale of growth. For example, an individual SDC might want to grow much more than what the SP has planned for the entire system. An aligned objective is where the growth ambitions of SP subsume the growth ambitions of the SDCs.

The growth objectives for SP and SDCs are defined in terms of the desired capacity being a multiple of the initial capacity for a skill. We have considered the case when the growth multiple is uniform across the SDCs. The first policy that we study is biased towards small SDCs. That is, the quantum of work to be done to accumulate credit to increase capacity is smaller for small SDCs compared to the large SDCs. The second policy that we study is a non-biased policy with a small bias against SDCs that grow too big. The experiment is carried out by fixing a policy and the work allocation mode at a time and then carrying out the simulations by varying the 3 parameters mentioned above and measuring the average time that each SDC takes in

224224224210210

meeting its objectives. We found a relation between the time taken and the SDC sizes for all types of workloads that were considered. The results are summarized in TABLE IV. The results below are true for all mixes of the SDCs in the system. That is, we took a system with very large number of large SDCs, another with very large number of small SDCs and so on, and the following observations hold irrespective of the mix. It is also noteworthy that the conflicting and aligned objectives showed similar behavior when the work was distributed purely through auction, that is, pure pull mode. The experiments for pure push mode with conflicting objectives did not yield interesting results, as in most cases the SDCs were unable to meet their local objectives.

TABLE IV. PULL FOR ALIGNED AND CONFLICTING OBJECTIVES

Time taken by SDCs (classified according to size) to meet objectives

Ratio of Work Packets

(small:medium:large)

Policy biased for small SDCs

Non biased policy with bias against SDCs that

grow too big

1:0:0 small<medium<large large<small<medium

0:0:1 medium<small<large large<medium<small

1:1:1 small<medium<large large<small<medium

The results for pure push in TABLE V for the third column should be read as: all the large SDCs in the system are split into two sets large1 and large2 such that one of the sets (large1) performs best while the other performs the worst.

TABLE V. PUSH FOR ALIGNED OBJECTIVES

Time taken by SDCs (classified according to size) to

meet objectives Ratio of Work

Packets (small:medium:larg

e)

Policy biased for small SDCs

Non biased policy with bias against SDCs that

grow too big

1:0:0 small<medium=large large1<small<medium<large2

0:0:1 large1<small=medium<large2

large1<small=medium<large2

1:1:1 large1<small<medium<large2

large1<medium<small<arge2

The results show that policies and the size of SDCs do play an important role in the time that is taken by an SDC to meet its objective. Moreover, this role is more pronounced in the pull mode than the push mode. Result Analysis: The pure pull depends on the policy because the bid amount is inversely proportional to the quantum of work that remains to be done to move to the next quantized level as described in the policy. When the policy is biased for small SDCs, then small SDCs have a higher bid amount as they have to do less work to be able to move to next level. The large SDCs will have to do lot of

work for same gains due to which they may always end up with comparatively smaller bids. When the policies are non-biased, then large SDCs have higher bid amounts because they have a favorable ratio of (work volume*work duration)/(quantum of work hours required). The small SDCs lose out when the workload consists of only large packets because once they acquire large packet, they do not have capacity to carry out other packets for a long duration due to which they take longer to meet the objectives. The medium SDCs lose out in general because large work packets have affinity for large SDCs and small packets for small SDCs and medium work packets can be handled by large SDCs well. The pure push seems to be independent of the policies. The fact that some large SDCs do well and some do not can be attributed to the fragmentation in the capacity of the system (total remaining capacity is greater than the size of the work packet, but the remaining capacity of each SDC is less than the size of the work packet). Some of the large capacities skills present in the system are always able to get higher rank either by virtue of load balance or by virtue of bigger increase in capacity by acquiring enough credits. Since large work packets result in higher credit amount, an SDC may be able to conjure credits just by doing a large work packet and since large SDCs will handle large packets more often, it may so happen that they get enough work soon enough to actually meet the local objectives very fast. However, for other large SDCs that do not do well, it happens because they are assigned medium work packets which fragment it in a manner that it is not able to free up enough capacity for a large work packet. This is verified by the case when we have only small work packets in the workload. In the case of non-biased policy, some of the large SDCs do well because they are able to grow more for same amount of work done. This is because the growth is in percentage terms rather than absolute values. The fact that a large SDC can handle more number of larger packets at a time compared to others also nullifies the effect of the long duration associated with a large packet. Small SDCs perform better or equally well as medium ones in most cases because they are assigned small or medium packets which finish comparatively faster and hence they quickly acquire the credits to meet the local objectives whereas the chances of a medium SDC getting stuck with few large packets is more which slows down the rate of reaching the objective. We finally carried out the experiments with pull+push mode and found that the average time taken by the SDCs comes down. This mode balances the extremes that are seen in the pull mode. Experiment V: Evaluate the role of the simulator as an advisor to an organization designer Given a fixed number of SDCs that have to be built into the system with flexibility in the sizes of SDCs, an important

225225225211211

question that an organization designer has to address is what should be the initial sizes of the SDCs. To see if the simulator can be of some use, we carried out an experiment where we fixed the number of SDCs in the system as 20 and such that 10 SDCs had same set of 3 skill types while the other 10 also had the remaining 3 skill types (recall that we have 6 skill types in our simulation set up). The simulations were run for a fixed set of workload on various configurations of these 20 SDCs. The configuration w.r.t. objectives and work allocation strategy that were used are: Misaligned objectives (local objectives more than global objectives), push+pull mode and push mode. The simulations were then run for various system configurations (mix of small, medium and large SDCs) for each of the above two configurations. The results for the time taken to meet SP objectives are shown in the tables below.

TABLE VI. PUSH BASED ALLOCATION

Time taken to meet SP objectives for various skills

Ratio of SDC sizes (small:medium:larg

e) java tivoli lotus Db2 unix C++

1:1:8 422 1182 1194 1144 1202 1268

8:1:1 262 476 723 632 707 517

3:3:4 332 857 796 891 850 870

TABLE VII. PUSH+PULL BASED ALLOCATION

Time taken to meet SP objectives for various skills

Ratio of SDC sizes (small:medium:larg

e) java tivoli lotus Db2 unix C++

1:1:8 442 702 853 761 783 795

8:1:1 184 421 404 424 449 376

3:3:4 286 413 566 431 598 509

TABLE VIII. PULL BASED ALLOCATION

Time taken to meet SP objectives for various skills

Ratio of SDC sizes (small:medium:larg

e) java tivoli lotus Db2 unix C++

1:1:8 446 861 853 817 783 795

8:1:1 183 431 428 427 455 414

3:3:4 312 533 571 561 606 552

It can be seen that in all the cases of work allocation, a system with many small SDCs appears to be doing best in terms of meeting global objectives. Thus, based on the simulator’s result analysis, the organization designer may decide to populate the system with a larger number of small SDCs, while retaining a smaller number of medium and large SDCs, to help reach the global objectives quickly.

C. Experimental Findings. We summarize the main observations from our experimental studies below:

1) Auction mechanism (pull mode) is important to sustain a system with autonomous SDCs.

2) A combination of push and pull modes results in the most balanced evolution of the system.

3) The evolution policies and size of the SDCs play a decisive role in the rate at which the SDCs reach the desired state. Specifically, we observe that the aided-push has to be used judiciously.

4) The presence of a giant SDC can affect the overall system. If the local objective of the giant SDC is orthogonal to the global objective, then, the system evolves slower as compared to systems in which they are aligned.

V. CONCLUSIONS AND FUTURE WORK We studied the factory model for service delivery in which the SDCs have limited autonomy to pursue local objectives. This was enabled in the form of the pull mode of work allocation via an auction mechanism. We obtained some insights on the design of the factory model. Firstly, the combination of push-pull allows the system to evolve in a manner conducive for global and local objectives. The allocation of budget to the SDCs can be used to influence weaker SDCs via the aided-push mode. But, it has to be done judiciously so as to not affect the other SDCs. The presence of a giant SDC can affect the evolution of the overall system. Its impact is positive when its objectives are aligned with the global objective. These insights, as well as simulation runs that compare the evolution of the system under different combinations (ratios) of small, medium and large SDCs, can help practitioners to design the appropriate factory model of service delivery. There are many interesting research directions for future work. We name a few. We studied the model in which work packets arrive one by one. An interesting question is, if the evolution of the system gets affected (and in what ways) if both the SP and the SDCs have limited look-ahead on the arriving work packets. In our model, we assumed that the SDCs do not have any visibility on the local objectives of others SDCs. It would be interesting to study how the behaviour of the SDCs and hence that of the system change, if we assume that the SDCs have limited visibility of the local objectives of the other SDCs. Our work indicates that there is a relationship between the performance of an SDC

226226226212212

and its characteristics such as size (relative to other SDCs), skill profile (relative to other SDCs), and local objectives. Let us define the resiliency of an SDC as its ability to deal with the changes in the workload characteristics (it is called drift or conceptual change in the time-series literature) in a competent manner. An interesting research direction is to study how characteristics of an SDC such as size, skill profile, and local objectives affect its resiliency.

REFERENCES

[1] A Garbage Can Model of Organizational Choice. M.D.Cohen, J.G.March and J.P.Olsen. Administrative Science Quarterly, Vol. 17, No. 1 (1972), pp 1-25

[2] Exploration and Exploitation in Organizational Learning. J.G.March. Organization Science, Vol. 2, No. 1 (1991), pp 71-87.

[3] Simulation Modeling in Organizational and Management Research. J.R.Harrison, Z. Lin, G.R.Carroll, K.M.Carley. Academy of Management Review, Vol 32, No. 4 (2007) pp 1229-1245

[4] Optimal Structure, Market Dynamism and the Strategy of Simple Rules. Davis, Jason, Eisenhardt and Bingham. Administrative Science Quarterly, 54 (2009): pp 413-452.

[5] Investing in Capabilities: The Dynamics of Resource Allocation. Organization Science, Articles in Advance (2010), pp 1-19.

[6] On the Design of Hierarchies: Coordination versus Specialization. O Hart and J. Moore. Journal of Political Economy Vol 113, Issue 4 (2005), pp 675-702.

[7] Intelligent Agent Based Workforce Empowerment. N. Shah, E. Tsang, Y. Borenstein et al. Agent and Multi-Agent Systems: Technologies and Applications. LNCS Vol. 5559 (2009), pp. 163-172

[8] An Agent-Based Approach to Improving Resource Allocation in the Dutch Youth Health Care Sector. E. Giesen, W. Ketter and R. Zuidwijk. 17th European Conference on Information Systems (2009)

[9] S. Ko and G. Loo, “Li and Fung: Growth of a supply chain specialist”, in Harvard Business Review, 2009.

[10] I. Musio, “IBM Industry Practise: Challenges in Offshore Software Development from a Global Delivery Center,” In proceedings of the Software Engineering Approaches in Offshore and Outsourced Development (SEAFOOD), 2009.

[11] R.L. Graham, E.L. Lawler J.K. Lenstra and A.H.G.Rinnooy Kan, “Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey,” Annals of Discrete Maths, vol. 5, 1979, pp. 287-326.

[12] Peter Brucker, Andreas Drexl, Rolf Möhring, Klaus Neumann and Erwin Pesch, “Resource-constrained project scheduling: Notation, classification, models, and methods,” European Journal of Operations Research, vol. 112(1), 1 Jan 1999, Pages 3-41

[13] W. S. Herroelen and E. L. Demeulemeester, “Resource-constrained project scheduling: a survey of recent developments,” Computers and Operations Research, vol. 25, 1998, pp. 279-302

[14] Paul Milgrom, “Putting Auction Theory To Work”, Cambridge University Press, UK, 2004.

[15] Prasad M. Deshpande, Dinesh Garg and N. Rama Suri, “Auction Based Models for Ticket Allocation Problem in IT Service Delivery Industry”, IEEE International Conference on Services Computing, vol. 1, pp. 111-118, 2008.

227227227213213