Ms

VU

a

ARRAA

KMSSM

1

isrgotpbcoa

acnmetu

h1

Applied Soft Computing 23 (2014) 165–179

Contents lists available at ScienceDirect

Applied Soft Computing

j ourna l h o mepage: www.elsev ier .com/ locate /asoc

ulti-agent based approach for single machine scheduling withequence-dependent setup times and machine maintenance

ahit Kaplanoglu ∗

niversity of Gaziantep, Department of Industrial Engineering, Gaziantep, Turkey

r t i c l e i n f o

rticle history:eceived 12 April 2013eceived in revised form 17 June 2014ccepted 17 June 2014vailable online 25 June 2014

eywords:ulti-agent based optimization

a b s t r a c t

Scheduling of single machine in manufacturing systems is especially complex when the order arrivalsare dynamic. The complexity of the problem increases by considering the sequence-dependent setuptimes and machine maintenance in dynamic manufacturing environment. Computational experimentsin literature showed that even solving the static single machine scheduling problem without consideringregular maintenance activities is NP-hard. Multi-agent systems, a branch of artificial intelligence providea new alternative way for solving dynamic and complex problems. In this paper a collaborative multi-agent based optimization method is proposed for single machine scheduling problem with sequence-

ingle machine schedulingequence-dependent setupachine maintenance

dependent setup times and maintenance constraints. The problem is solved under the condition of bothregular and irregular maintenance activities. The solutions of multi-agent based approach are comparedwith some static single machine scheduling problem sets which are available in the literature. The methodis also tested under real-time manufacturing environment where computational time plays a critical roleduring decision making process.

© 2014 Elsevier B.V. All rights reserved.

. Introduction

Scheduling problems are often found in various businesses andndustrial application areas [1]. Real world scheduling problemsometimes differ from the ones for which there exist many algo-ithms to solve. However, real world scheduling problems areenerally much more complex than the ones being solved in the-ry. Job scheduling problem is one of the instances. It belongs tohe well-known combinatorial optimization problem domain. Inractice, coordination of the jobs and machines under complexusiness environment is a challenge. Some of the sources of diffi-ulties could be given as; machine non-availability, unexpected eventccurrence, sequence-dependent setup times and unpredictable orderrrivals [2–8].

Machine non-availability: It is generally assumed that machinesre available at all times. However, in practice machines are notontinuously available. There are many reasons why machines mayot be in operation. Some of the reasons are based on a deter-inistic process, others on a random process. The most commonly

ncountered type of machine non-availability is preventive main-enance activities. As a complex system with multiple componentssually deteriorates with age, preventive maintenance is often

∗ Tel.: +90 3423172610; fax: +90 3423604383.E-mail addresses: kaplanoglu@gantep.edu.tr, vahitkaplanoglu@hotmail.com

ttp://dx.doi.org/10.1016/j.asoc.2014.06.020568-4946/© 2014 Elsevier B.V. All rights reserved.

performed to keep the system functioning in a good state to prolongits effective age [3]. Preventive maintenance tasks are performedwhen the machines are idle and it represents one of the determinis-tic sources for machine non-availability [4]. In some manufacturingenvironments, there may be a need for maintenance before themaintenance program is reached due to aging factors of jobs. Thisproblem type is similar to capacitated vehicle routing problemwhere the vehicles must return to depot after loading/unloading oftheir loads. In such conditions both programmed maintenance andun-programmed maintenance activities must be considered simul-taneously. Whenever total aging level of the machines reachesto a point, then the machines require maintenance. In this paperit is called maximum aging capacity of the machines. Each jobconsumes the aging capacity of the machine according to its pro-cessing characteristic. Maintenance activity is performed whenmaximum aging capacity or the programmed maintenance isreached.

Unexpected event occurrence: In most real-world manufac-turing environments, scheduling is an ongoing reactive processwhere the presence of a variety of unexpected disruptions isusually inevitable. Unexpected disruptions continually force recon-sideration and revision of pre-established schedules [5]. Order

cancelations and machine breakdowns are two commonly encoun-tered sources of unexpected events in machine scheduling. Thecomplexity of the problem increases by considering the revisionand reconsideration of any pre-established machine schedule.

1 t Com

mhos

mpt

aplbo

aIbimtatimmattciatigmtt

attet

2

tidjasacsaitrc

j

66 V. Kaplanoglu / Applied Sof

Sequence-dependent setup times: In manufacturing environ-ents setup of the machines while changing from a job to another

as also a considerable effect on completion time. The complexityf the problem increases by considering the sequence-dependentetup time [7].

Unpredictable order arrivals: Some of the orders may arrive toanufacturing system while the system is operating [8]. The com-

lexity of the problem increases by considering dynamic arrival ofhe orders to the manufacturing environment.

Real manufacturing environments could face with all theseforementioned factors and some others such as uncertainty of jobrocessing time which are not in the scope of this paper. The prob-

em of single machine scheduling with maintenance is proved toe NP-hard even though the sequence-dependent setup times andther factors are not considered [9].

Multi-agent systems, a branch in artificial intelligence, provide new way for solving distributed, dynamic and hard problems.n artificial intelligence an agent is defined as anything that cane viewed as perceiving its environment through sensors and act-

ng upon that environment through actuators [10]. This capabilityakes multi-agent software entities a good candidate to handle

he distributed, dynamic and complex problems. In this paper, multi-agent based approach where each block between main-enance activities and jobs are designed as autonomous agentss proposed. The proposed approach enables scheduling single

achine when arrivals of jobs are dynamic without a need of re-odeling and re-solving the problem centrally. Distributed and

utonomous mechanism of blocks and jobs could be thought ashe contribution of this paper where it enables a dynamic and real-ime scheduling. The approach uses the flexibility of decentralizedontrol systems with its distributed block and job agents. A mixed-nteger linear model is solved by the agent types defined in thispproach. Therefore the agents in this present approach supportheir decision with optimization method. The agents consider theirntegral part of local optimization problem. In this paper, the mainoal is to find fast, adaptive and effective solution to dynamic singleachine scheduling problems. The proposed approach uses opera-

ions research (OR) and multi-agent system methods to cope withhe dynamic nature of machine scheduling problem.

The paper is organized as follows: in Section 2, state of the artbout this domain is given. Section 3 introduces multi-agent sys-ems, Section 4 shows the problem definition, Section 5 presentshe proposed approach, Section 6 shows some computationalxperiments for the proposed approach and Section 7 summarizeshe paper.

. State of the art

Single machine scheduling problem has attracted the atten-ion of many researchers. There is a wide variety of research arean single machine scheduling problem which spans from job dueates consideration to past-sequence-dependent delivery times of theobs where the adverse effects of waiting are incorporated into

post-processing operation by introducing the concept of past-equence-dependent (p-s-d) delivery time [11,12]. However, therere rare studies for single machine scheduling problem whichonsider machine maintenance. There is not a study which con-iders single machine scheduling problem together with regularnd irregular maintenance activities. In many of the previous stud-es researchers generally considered single maintenance activityhroughout the entire manufacturing process. The proposed algo-

ithms face difficulties for the long term scheduling due to theirentral structure.

Chen [13] dealt with the problem of scheduling non-resumableobs on a single machine with flexible and multiple maintenance

puting 23 (2014) 165–179

activities. If a job cannot be completed before the unavailableperiod, then it must be restarted rather than continued, a casecalled non-resumable. The resumable case is that where a jobcan continue after the machine becomes available again. Thereare also some studies which consider scheduling problems withat most one maintenance period [14–16]. Low et al. [17] consid-ered the availability constraint of job deterioration. Mosheiov andSarig’s research [18] on the problem of scheduling concentrates on amaintenance activity in a single machine with the objective of mini-mizing the total weighted completion time. There are some studiesfor machine scheduling which consider the maintenance cost aswell [19]. In their paper, Sarker et al. [2] proposed a hybrid evolu-tionary algorithm for job scheduling under machine maintenance.They proposed a reactive scheduling algorithm under expected andunexpected machine unavailability. They considered unexpectedmachine unavailability in their paper. However, they used a centralsearching algorithm where re-schedule is done centrally.

Multi-agent based systems are rarely employed for manufac-turing environments and machine scheduling domain. One of therare studies in this domain is presented by Khelifati and Benbouzid-Sitayeb [20]. They proposed a distributed approach which is usingmulti-agent paradigm for scheduling independent jobs and main-tenance operations in the flow-shop sequencing problem. Qin andLi [21] used ILOG and multi-agent based framework for job-shopscheduling system. Duan et al. [22] proposed a negotiation-basedoptimization method for scheduling of a manufacturing system.There are two main agent types in their paper which are man-ufacturers and suppliers. In their paper, Erol et al. [23] proposedmachine scheduling together with the automated guided vehicles(AGVs) in a flexible manufacturing environment. In their paper,Lou et al. [24] presented a multi-agent-based proactive-reactivescheduling for job shop scheduling problem. Hercog [25] proposeda better manufacturing process organization using multi-agent sys-tems. Polyakovskiy and M’Hallah [26] proposed a heuristic which isbased on multi-agent system to solve weighted earliness tardinessparallel machine problem. Henchiri and Ennigrou [27] proposed amulti-agent model based on hybridization of tabu search methodand particle swarm optimization in order to solve flexible job shopscheduling problem (FJSP). Wang and Chen [28] designed a multi-agent scheduling mechanism based on an iterative combinatorialauction to deal with parallel machine scheduling with controllableprocessing times. Grimme et al. [29] presented an agent-basedapproach to multi-criteria machine scheduling problem. Savinoet al. [30] proposed an agent system for a multiple-objective flow-shop scheduling problem in a production context characterized bydiversified, high-volume production mix.

Chen and Wang [1] suggested a collaboration framework ofan agent-based intelligence system that can handle the problem-solving of distributed project in real situations. Their paper is one ofthe most relevant papers in the literature. In their paper, they useda two-stage decision-making process applied to dynamic schedul-ing in the system through the fuzzy decision-making. They usedmulti-agent system paradigm for dynamic multi-contract projectscheduling where there exist several independent, specialized anddistributed subcontractors during project management process.The model that they offered is similar to the proposed approachgiven in this present paper; both of the papers consider solvingdynamic scheduling problem by using multi-agent systems. How-ever, in their study there is more than one party (general contractor,subcontractors and their subcontractors) in the model. The systemis used to control and resolve the difficulties for project partici-pants in distributed projects, especially focusing on the dynamic

solving of the scheduling problem. Fuzzy decision-making and thecompensatory negotiation processes are used by the agent types.The methodology of Chen and Wang [1] and of this present papercan maintain a high degree of openness and flexibility due to their

V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179 167

Maintenance Maintenance

1 2 3 4 5

Job insertion

j1 j2 j3 j4 j5

Maintenance

Job insertion

T

Time

amic

pgifssdt

3

loclcttowi

iotfasc

MwacMtdbdMwTm

•••

•

just before job j.si0 = preparation time before maintenance.s0j = preparation time after maintenance.wi = aging weight of job i.

Fig. 1. Dyn

rogramming structure. Both studies use Java programming lan-uage and its standards as agent programming language. However,n this present paper, the reasoning of job assignment is designedor single-machine within a manufacturing environment. There areome logical parties which collaboratively try to find a dynamicchedule for single machine scheduling problem with sequence-ependent setup times and machine maintenance activities whilehe system is operating.

. Multi-agent systems for scheduling domain

Nowadays, agent systems are used to resolve scheduling prob-ems in some work environments. Agent systems can be used asnline scheduling methods [31]. An agent is a type of software thatan autonomously perform routine tasks with a degree of intel-igence [32]. Agents are also considered as social, because theyooperate with humans or other agent types in order to achieveheir tasks. Additionally, agents are reactive, because they perceiveheir environment and respond in a timely fashion to changes thatccur in the environment. When an agent is instantiated, it willait until it is given a goal to achieve or experiences an event that

t must respond to [33].When many agents operate as a collaborator or competitors,

t is called multi-agent systems (MAS). MAS are a collection ofrganized, intelligent and autonomous software agents that workogether or compete to find answers to problems that are complexor a single software agent to solve [23]. There are so many differentpplication areas of MAS from supply chain management to humanpatial behavior modeling, unmanned vehicle control systems andloud systems [34–36].

MAS can also be used as online scheduling methods [37].achine scheduling domain seems very suitable for agent researchhen all the definitions and characteristics of agent and/or multi-

gent based systems are considered. Machine scheduling domainould be considered as decentralized, distributed and dynamic.AS have become increasingly popular among decentralized con-

rol systems since these systems are designed to operate underistributed and autonomous business conditions [38]. MAS coulde considered for controlling job orders in a dynamic, volatile andistributed manufacturing environment. It could be stated thatAS can help to solve dynamic single machine scheduling decisionshen regular and irregular maintenance activities are considered.

he reasons for why multi-agent based systems are suitable forachine scheduling domain could be listed as:

The domain is inherently distributed

Human-experience (intelligence) is used for planning operationsHigh amount of environmental factors exist (worker unavailabil-ity, machine failure, rush jobs, job cancelation, due date changes,etc.)Need for a cooperation among the parties

job arrival.

Organizational units like planners, machines, jobs, blocks, etc.,may be represented as agents. After representing them as an agenttype the capabilities of agent systems could be used to makescheduling units to compete, cooperate and find optimal (or nearoptimal) solutions in a dynamic way.

4. Problem definition

In this research domain there is an undefined set of jobs arriv-ing to a single machine manufacturing environment. Before a jobis processed on a machine a setup time is required. Setup timedepends on the previously processed job on the machine. Addi-tionally, regular and irregular maintenance activities are requiredfor the machine. Whenever total aging of the machine reaches toa defined level then machine requires maintenance. In this paperit is called maximum aging capacity of the machine and notated byW. Periodic maintenance is executed for every T time units. Blockterm is used to represent time interval between two consecutivemaintenance activities whether regular or irregular. Fig. 1 showsthe problem structure.

Fig. 2 shows an example of a block schedule. The schedule isobtained separately for each block available in the system whetherits length is T or smaller than T. The notation Lm is used for thevariable-size block length, where Lm ≤ T. Time slots between jobsshow the setup times between any consecutive jobs. It is also pos-sible that there may be idle slots before a maintenance activity.

The characteristics of the problem are as follows:

• There are indefinite number of jobs arriving to the manufacturingenvironment

• Each job j has an associated processing time pj• Each job j has an associated setup time based on its predecessor• The jobs cannot be interrupted• Time periods between maintenance activities (Lm) are variable in

their length• The number of blocks used in the schedule is unknown

The notations used in this research are as follows:

Jj = job number j.pj = processing time of job j.sij = setup (preparation) time that depends on the job i processed

Maintenance Maintenance1 23 4 5Time

Fig. 2. Example schedule of a particular time block.

168 V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179

Maintenance Maintenance

W

btcata

jap

5p

iitbrltAiimpBt

Table 1Agent type definition.

Job Dispatcher AgentType

Job dispatcher agent type creates job agentsbased on the jobs which are arriving to systemdynamically. Its function is to create job agentsand release them to the system to interact withthe machine block agents. Unexpected eventsrelated with jobs are also handled by this agenttype. It turns the unexpected events into amessage and sends them to relational jobagents.

Resource ManagerAgent Type

Resource manager agent type controls thecreation of machine block agents when it isneeded. Resource manager agent also sendsthe message of breakdown and reparation tojob agents in the case of a breakdown andreparation respectively.

Machine Block AgentType

This agent type is the surrogate of thevariable-length time (Lm) block between twoconsecutive maintenance activities. It behaveson behalf of time-block in the realmanufacturing system.

Job Agent Type Job agent type is the agent type that stands forthe dynamically arriving jobs. Job agents arecreated by the job dispatcher agent after they

T

Fig. 3. Maintenance capacity representation.

W = maximum aging capacity of a machine before maintenance.T = time interval between two consecutive regular maintenanceactivities.

An oriented graph is used to define the problem. Let G = (V,A)e an oriented graph, V = {0,1,. . .,n}s are the set of nodes and A ishe related set of arcs. cij ≥ 0 is the associated weight of arc(i,j).ij is defined as sum of the time required to prepare the machinend to process the job just after job i. Then cij = sij + pj. Fig. 3 trieso show the meaning of parameter W which holds the maximumging capacity of a machine before a maintenance activity.

The objective could be given as; to assign dynamically arrivingobs to the blocks between regular and/or irregular maintenancectivities in such a way that the last job on hand finishes as soon asossible that is, minimizing Cmax.

. Multi-agent based approach for machine schedulingroblem

Agent architectures are the fundamental mechanisms underly-ng the autonomous components that support effective behaviorn real-world, dynamic and open environments. Agent architec-ures can be divided into four main groups: logic based, reactive,elief-desire-intention (BDI) and layered architectures [39]. In thisesearch BDI architecture is used to model the proposed prob-em because there are many effective available technologies forhis architecture such as JACKTM. The JACKTM Agent Language is angent Oriented programming language and is used for implement-

ng agent oriented software systems. It also has its own modelingnterface which runs under BDI architecture. Following the BDI

odel, JACKTM intelligent agents are autonomous software com-onents that have explicit goals to achieve and events to handle.DI agents are programmed with a set of plans in order to achievehese desires. Each plan describes how to achieve a goal under

Fig. 4. Proposed multi-agent based

are accepted by the system.

varying circumstances. Set to work, the agent pursues its givengoals (desires), adopting the appropriate plans (intentions)according to its current set of data (beliefs) about the state of theworld. This combination of desires and beliefs initiating context-sensitive intended behavior is a part of what characterizes a BDIagent [40].

The proposed approach consists entirely of BDI agent types andBDI agent types are implemented on JACKTM. Each agent type surro-gates real-life components of machine scheduling. Blocks betweenmaintenance activities, jobs, job dispatchers and resource man-agers are represented as BDI agent types in the proposed approach.Unexpected events are represented as event types. Top-most viewof the proposed multi-agent based approach is given in Fig. 4 andagent type definitions are given in Table 1.

After introducing the agent types, their conceptual designs andimplementation on JACKTM are illustrated in this section of thepaper. Fig. 5 depicts the flow diagram of the proposed approach.

approach overview diagram.

V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179 169

Is there anymachine block

agent?

NoYes

Processing of job agent

Arrival of a new job agent tomanufacturing system

Is there any slacktime for job agent?

Yes

No

Finish

Create a new machine block agent

Update machine schedule

Negotiation between job and machineblock agent(s)

Fig. 5. Flow diagram of multi-agent based approach.

Fig. 6. Job dispatcher agent d

Fig. 7. CreateJobAgentPlan editor view.

5.1. Job dispatcher and resource manager agent types

Whenever a new job is received from the manufacturing sys-tem a new job agent is dynamically created by the job dispatcheragent. Its function is to create job agents and release them to systemto interact with machine block agents. The information about jobwhich is dynamically arriving to manufacturing system is pack-aged and sent to the job agent after creating it. Fig. 6 shows jobdispatcher agent design view on JACKTM agent development envi-ronment. Fig. 7 shows plan editor of JACKTM for CreateJobAgentPlanof job dispatcher agent.

Job dispatcher agent provides a list of machine block agents tojob agents. It has a list of machine block agents on its beliefset andprovides a list to the newly created job agents. Job agents negotiatewith machine block agents according to list provided by the jobdispatcher agent.

Resource manager agent controls the creation of machine blockagents. When job agents are rejected by all of the machine block

esign view in JACKTM.

170 V. Kaplanoglu / Applied Soft Com

aa

5

pMjaa

baaajdinra

simbt

sm

a

InsertToEndOfTheBlock. These plans are decided by the parameters

Fig. 8. Job agent and machine block agent interaction protocol.

gents then a new machine block agent is created by resource man-ger agent.

.2. Machine block agent type

The goal of the machine block agent is to respond to call forroposals (cfp) of job agents with a minimum completion time.achine block agent can make its own reasoning while making the

ob assignment decision. Machine block agent sends its decisionbout assignment to the related job agent. Fig. 8 shows the inter-ction protocol between machine block agent and job agent types.

When a job agent j enters to the system it has to bid to machinelock agents for a call for proposal (cfp). As presented in Fig. 8, the jobgent entering to system calls for a proposal to the machine blockgents which are provided as a list by job dispatcher agent. If therere m numbers of machine block agents in the list provided thenob agent calls for proposal to these m blocks within the system. nenotes the number of machine block entities which are respond-

ng to the call for proposal. i is the number of rejection, a is theumber of machine block agents which accepted the cfp. k is theejection number and l is the number of acceptance of job agents-base case.

After the acceptance of a job agent by a machine block agent,chedule of the respective block is re-built. The proposed approachs designed for dynamic and open systems where job cancelations

ay occur. The proposed system supports re-scheduling of thelocks and job change mechanism. Therefore machine schedule isemporary till the end of processing of all jobs on machine.

The negotiation protocol is implemented on JACKTM. Fig. 9

hows the implementation of interaction protocol between job andachine block agents.Job agents use instructions given below to start negotiation of an

ssignment. The assignRequestET event type carries the data about

puting 23 (2014) 165–179

the job agent to the machine block agent. Job agent uses the fol-lowing instruction to make a cfp to machine block agent.postEvent(assignRequestET.request(mb[i],j));vectorList.add(mb[i]); numberOfSent++;

In this instruction, mb[i] parameter holds the address data of themachine block agent. Machine block agent evaluates the proposalof job agent after it is called for proposal with assignRequestET event.Reasoning mechanism of machine block agent is given in Fig. 10.Machine block agent runs four different plans simultaneously. Itselects one of the plans based on its reasoning.

Fig. 10 shows four different plans of machine block agent. Theplans are; Optimize, ApplyMetaHeuristics, ApplyRandomSearch andInsertToEndOfTheBlock.

The Plan class in JACKTM describes a sequence of actions that anagent can take when an event occurs. Whenever an agent is postedan event, it tries to find a plan to handle the event. Plans describe, inexplicit detail, exactly what an agent should do when a given eventoccurs. Equipped with a set of plans, an agent has a set of skills andprocedural knowledge that it can draw upon as required. When theevent that a plan addresses occurs, the agent can execute this planto handle it. Each plan is capable of handling a single event. Theevent is identified by the plan’s #handles event declaration [40].

Multi-agent approach gives reasoning capability to the entitiesdefined in the system. Machine block agent could change its com-putation method while it is processing a call for proposal. Fig. 11shows possible states of a machine block agent. Machine blockagent could be in states of; waiting for call for proposals, com-puting a previously accepted job agent request, interrupted with anewly arriving job agent cfp or applyNewPlan. While in any state,machine block agent could shift to another state. The states arecalled threads in computer programming. Fig. 11 is showing theshifting mechanism between the threads of execution. As an exam-ple, let’s assume machine block agent is in computing state thenmachine block agent could select optimize, applyMetaHeuristics,applyRandomSearch or insertToEndOfTheBlock plans. If optimize planis selected then machine block agent starts applying this plan.During optimize plan is running, a new job agent request mightinterrupt machine block agent. In this case, machine block agentcould change its plan from optimize to applyMetaHeuristics becauseof computation time consideration. This mechanism is contributingmachine block agent to shift from optimize plan to applyMeta-Heuristics plan without losing any computational time. JACKTM isproviding plan editor to define shifting conditions from one plan toanother (see Fig. 10).

Machine block agent makes its reasoning based on relevant( ),context( ), body( ) and fail( ) methods. Machine block agent posts thefollowing message to itself to run the optimize plan as following:@post(optET.OPTIMIZE(ev.data));

Machine block agent makes its reasoning with context( ) methodto make the decision of optimization. If the context( ) methodapproves the relevance of the method then body( ) method askoptET event instance to find the optimum completion time formachine block agent by using a pre-packaged mathematical solver.

Machine block agent holds the jobs data which are previouslyaccepted. Fig. 12 represents list of previously accepted job agentsby machine block agent.

Each machine block agent holds the data about jobs in a 4-tupledata structure which are job processing time (p), job set-up time(s), aging factor of job agent (w) and effect of job agent to blockentity (e).

Machine block agent makes its own reasoning to decide whetherto choose Optimize, ApplyMetaHeuristic, ApplyRandomSearch and

given before the program run. If the number of job agents in theschedule of machine block agent exceeds parameter � then Apply-MetaHeuristic plan is applied. If number of job agents in schedule

V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179 171

d ma

odpI

Fig. 9. JACKTM implementation of job an

f machine block agent exceeds the parameter then ApplyRan-

omSearch plan is applied with a computation time parameter � , ifreviously accepted job agents are fixed with their schedule then

nsertToEndOfTheBlock plan is used by the machine block agent. This

Fig. 10. Machine block agent jo

chine block agents interaction protocol.

plan is executed when machine block agent is requested to fix the

job schedules (if required in real-time execution of the program).

Whatever the plan selected machine block agent calculates theeffect of job agent (ej) which is calling for proposal. The following

b assignment reasoning.

172 V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179

MachineBlockAgent

waiting computing interruption

new JobAgent

trigger

newevent

wa ittil l

newJ ob

Agentcfp

event

drive

interruption

info

applyNewPlan

interruption

newplan

endof

interruption

agent

ef

e

wiaa

aa

5

sotj

x

u

T

m

∑

∑

Fig. 11. Machine block

quation is used to calculate the effect of a job agent which is callingor proposal to a particular machine block agent.

ib = Cmaxib − Cmaxb (1)

here eib is the effect of job agent i to machine block agent b. Cmaxibs the completion time of machine block agent b after accepting jobgent i. Cmaxb is completion time of machine block agent b beforeccepting job agent i.

If it is not feasible for machine block agent to accept the jobgent calling for proposal, Eq. (1) is not run by machine block agentnd refusal message is sent to relational job agent (see Fig. 8).

.2.1. Optimize planMachine block agent tries to find its minimum Cmax value. It

olves the problem with aggregated jobs in its list upon running itsptimization plan. Machine block agent uses the following nota-ion and formulation in order to find the optimum sequence ofobs [41]:

ij ={

1, if arc(i, j) is used in the machine block agent

0, otherwise

i = auxiliary variable to schedule all the jobs in machineblock agent list

hen, the mathematical model of machine block agent is;

in z =n∑i

n∑j

cijxij

n

i=1

xij = 1 (for j = 1, 2, . . ., n) (Constraint 1)

n

j=1

xij = 1 (for i = 1, 2, . . ., n) (Constraint 2)

J1=(p1, s01, w1, e1b)

j1 j2

J2=(p2, s12, w2,

Fig. 12. Previously accepted job ag

thread execution cycle.

ui − uj + nxij ≤ n − 1 (for i /= j; i = 2, 3, . . .n; j = 2, 3, . . ., n)

(Constraint 3)

n∑i

n∑j

cijxij ≤ T (for i = 1, 2, . . ., n; j = 1, 2, . . ., n)

(Constraint 4)

n∑i

n∑j

wixij ≤ W (for i = 1, 2, . . ., n; j = 1, 2, . . ., n)

(Constraint 5)

All xij = 0 or 1, All ui ≥ 0 (Constraint 6)

cij and wi are the coefficients in the model. (Constraint 1) and(Constraint 2) impose the assignment of each job agent in themachine block agent list. (Constraint 3) eliminates the sub-toursin the model. (Constraint 4) enables scheduling to finish within thetime duration of T and (Constraint 5) enables the total weights ofthe jobs not to exceed aging capacity of machine block agent.

Machine block agent solves the mathematical model givenabove and if possible it finds the optimum sequence of jobs on itsown schedule.

5.2.2. ApplyMetaHeuristic plan: simulated annealing algorithmMachine block agent type could use optionally many different

computational algorithms in order to insert more job agents to itsschedule. The proposed multi-agent based approach is capable toarticulate any algorithm to its plans. The most advantageous part of

multi-agent based systems is their capability of extensibility. Anyalternative algorithm could be used (plugged) without making aserious change in the system. This makes the approach extensibleand open for improvement. Due to its effectiveness in scheduling

. . . jn

e2b)

Jn=(pn, sn-1 n, wn, enb)

ents in machine block agent.

V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179 173

j2 j3j1 j4 ... jn

job agent swapping

F

apm

pgapiaicwpwiioiawwgift

5atts

SS

5tis1b

T

T

wmt

jobAgent

negotiating waiting

wait

event

waitingtime

expires

assignment

processing

starting

eventnewarrival

orre-negotiateevent

and S is set of job agents in the machine block agent m schedule

ig. 13. Neighborhood move of a machine block agent schedule with n operations.

nd easiness to implementation, simulated annealing algorithm isreferred in this research as an algorithm to find schedule for eachachine block agent when optimize plan is failed.Ever since its introduction, based on the joint efforts by Kirk-

atrick et al. [42], simulated annealing (SA) algorithm has producedood results for large combinatorial optimization problems. SAlgorithm is very successful algorithm particularly in schedulingroblems [43]. It can be viewed as an analog of an algorithm used

n statistical physics for computer simulation of the annealing of solid to the state with minimum energy [44]. It has the capabil-ty of jumping out of the local optima for global optimization. Thisapability is achieved by accepting neighboring solutions which areorse than the current solution with a probability. The acceptancerobability is determined by a control parameter (Temperature)hich decreases during the SA procedure [45]. Starting from an

nitial solution, simulated annealing generates a new solution Snew

n the neighborhood of the original solution S. Then, the change ofbjective function value, � = f(Snew) − f(S) is calculated. For a min-mization problem, if � < 0, the transition to the new solution isccepted. If � > 0, then the transition to the new solution is acceptedith a probability, usually denoted by the function, exp(−�/T),here T is a control parameter called temperature. SA algorithm

enerally starts from a high temperature and then temperatures gradually lowered. At each temperature, a search is carried outor a certain number of iterations, called the epoch length. Whenermination condition is satisfied, the algorithm terminates [43].

.2.2.1. Neighborhood move. Each machine block agent uses SAlgorithm when the optimization plan is failed. In SA algorithm,he choice of neighborhood can greatly influence the success ofhe algorithm. In this work, neighbor solutions are generated by awapping procedure that is defined as follows:

tep 1: Select a job agent position randomly.tep 2: Determine a second position randomly from machine block

agent schedule and swap the job agents in these positions.

Swapping procedure is represented in Fig. 13.

.2.2.2. Cooling schedule. SA algorithm generally starts from a highemperature and then the temperature is gradually lowered. Thenitial temperature should be high in order to accept the generatedolutions randomly (by getting closer to acceptance probability of). Estimation for the initial and final value of the temperature cane made by using the following equations [45]:

in = (fmin − fmax)ln Pc

(2)

f = (fmin − fmax)ln Pf

(3)

here fmin and fmax are the lower and higher bounds for a givenachine block agent schedule, Pc is the initial probability of accep-

ance and Pf is the final probability of acceptance.

Fig. 14. Job agent thread execution cycle.

Temperature is reduced by a constant which is calculated withthe following equation of Bennage and Dhingra [46];

=(

ln Pcln Pf

)1/elmax−1

(4)

where elmax is the number of temperature reduction cycles (maxnumber of iterations). When the number elmax is reached then thealgorithm terminates. The final temperature could be calculatedwith Eq. (5). If final temperature is different from the one found inEq. (3) then modifications could be done for elmax [45]

Tf = Tin˛elmax . (5)

5.3. Job agent type

Job agent is an agent type that stands for the dynamically arriv-ing jobs. Job agents are created by the job dispatcher agent afterthey are accepted by system. Job agents send cfp to machine blockagents then wait till receiving all the responses from machine blockagents which are previously called for a proposal. It aggregates allthe responses and then finds the machine block agent which pro-poses the minimum effect value. It sends message of acceptance toone which proposes with minimum effect value and refuses pro-posal of the others (see Fig. 8). After doing that, job agent waitsuntil receiving the message of completion from its predecessor jobagent. Job agents are sent a message from resource manager agentwhen there is a machine breakdown. Job agents wait accordinglyin the case of machine failure. When the machine starts its oper-ation after reparation, resource manager agent sends the messageof operation to job agent.

After being accepted by a machine block agent, job agents sleepfor a time period during their execution lifecycle. They start negoti-ation with machine block agents after their sleep period. Sleep timefor job agents are represented with �. Job agent thread executioncycle could be seen in Fig. 14.

While waiting to its turn on a machine an order cancelationevent might occur (see Fig. 4). When an order is canceled by acustomer then the job dispatcher agent sends message of cancela-tion to canceled job agent. Canceled job agent runs its cancelationplan and sends machine block agent to update its schedule after itsremoval.

When a job agent is canceled, respective machine block agent isalerted about the effect values of the other job agents in its schedule.Let M is the set of available machine block agents in the system

then M = {m1,m2,. . .,mn} and S = {j1,j2,. . .,jt} where n is number ofavailable machine blocks and t is number of job agents in machineblock agent m.

1 t Com

jnam

hsouHe

arbnaatj2

6

tbacHdaccmopqtbn

piomBtabtIs

6u

wtip

b

74 V. Kaplanoglu / Applied Sof

Let jc be the canceled job agent in machine block agent m, thenob agents in set S\jc are updated with a new ej value and they re-egotiate with machine block agents in set M\m in order to find

better machine block agent. Job agents start negotiation withachine block agents autonomously.This capability of the proposed multi-agent based approach

elps manufacturing system to react without doing an entire re-cheduling. In dynamic case whenever an order cancelation eventccurs then the content of the relational machine block agent ispdated. Machine block agent runs its plans (optimize, applyMeta-euristics, applyRandomSearch or insertToEndOfTheBlock) to update

j values.Fig. 15 shows how job agents re-negotiate with machine block

gents when RemoveFromSchedule plan is executed. As a result ofe-negotiation it is possible to change job agents among machinelock agents. It results in a self-emerging scheduling where there iso need for central scheduling for all blocks and jobs. Fig. 16 showsn example change process between machine block agents after

series of job agent–machine block agent negotiations. There arewo different machine block agent schedules in Fig. 16. Job agents3 and j4 moved from machine block agent 1 to machine block agent

after re-negotiation.

. Computational experiment

Some benchmark tests are run in order to evaluate the contribu-ion of the proposed approach to single machine scheduling withoth regular and irregular maintenance activities under dynamicnd static circumstances. In general, optimal solutions can only beomputed if a problem is treated as a closed scheduling problem.owever, most of the real-industrial problems could be han-led with a dynamic model. Proposed agent based optimizationpproach could be used for both static and dynamic cases. In thease of static scheduling problem the proposed approach does nothallenge the full expressiveness of a dynamic system. The perfor-ance of the approach for static problems could be dominated by

ther central static algorithms. The biggest opportunity of the pro-osed system is its ability to response newly arriving job requestsuickly. When any job request is received by the agent based sys-em it is directly fit to a proper block agent. This advantage coulde called “living system” that the real manufacturing environmentseed.

Computational experiment section is divided into two mainarts: in the first part, the proposed multi-agent based approach

s tested with some static problem sets. In this part, two typesf experiments are conducted. First type of experiments considersachine scheduling without aging capacity. The results of Angel-

ello et al. [6] could therefore be compared with it. In the secondype of the experiment, the machine utilization weights of the jobsre also considered during the scheduling of the job entities onlocks. In the second part of the computational experiment section,he system is tested under dynamic manufacturing environment.n this part full expressiveness of the dynamic system is used tohow the capabilities of the proposed approach.

.1. Part I – computational experiments without consideringtilization capacity

In the first type of experiments, the machine scheduling is doneithout considering the aging capacity of the machine. W parame-

er in the mathematical model and simulated annealing algorithm

s given a big number in order to ignore the aging capacity. Onlyeriodic maintenance activities are considered.

Previous study of Angel-Bello et al. [6] is very helpful to showenchmarks for the proposed approach. The model proposed in

puting 23 (2014) 165–179

their paper was about scheduling machines with periodic main-tenance and sequence-dependent setup times. In their paper,Angel-Bello et al. [6] use asymmetrical vehicle routing problem(AVRP) set to test their heuristic approach. Test sets could beobtained from the link [47]. The first element in these problem setsis considered as the maintenance activity and loads are consideredas a job which will be processed on a single machine. The distancesbetween load pickup points are considered as cij for the test runsin terms of seconds.

In all the experiments, machine block agents use Gurobi 5.5Solver Package to find optimum schedules and simulated annealingas a meta-heuristic in their plans.

The T values which define the block length are determined byusing a dm parameter. dm is the maximum of the values of cij inthe problem set. Angel-Bello et al. [6] determined T values whereT ≥ 2dm. For different tightness levels T took values: 2.25dm, 2.5dm,3dm, 4dm.

To illustrate the performance of the proposed approach anexample solution could be given. In this example, simulated anneal-ing parameters are obtained as following; Pc and Pf are assumed as0.99 and 10−15, respectively. Value of (fmin − fmax) is assumed as−4000. elmax is determined as 5000. Tin and Tf are found as 397,996and 115.81, respectively, by using Eqs. (2) and (3). is calculatedas 0.998 (using Eq. (4)). The algorithm could be run for 5000 times.Sleep time for the job agents are all drawn independently fromuniform distributions with mean � and ranges [0,2�]. ApplyRan-domSearch plan of job agents are all drawn independently fromuniform distributions with mean � and ranges [0,2�]. �, , � and �are given 15, 30, 0.3 s and 0.1 s, respectively. Job agents are releasedto the system randomly during the run time.

The proposed approach finds its best for A034-2.25dm witha Cmax = 1340 in 2.517 s where T = 747 as given in the followingsequence.

Block 1: 0-16-15-14-12-13-9-32-7-4-6-5-30-26-27-28-29-25-1-33-2-3-0Block 2: 0-24-23-22-20-21-31-18-19-17-11-8-10.

In its worst run the proposed approach finds the followingsequence with a Cmax = 1532 in 2.152 s where T = 747.

Block 1: 0-25-24-23-27-28-29-26-22-20-21-18-17-11-8-9-12-13-0Block 2: 0-1-19-31-10-32-7-4-6-5-30-33-2-3-0Block 3: 0-16-15-14-0

Table 2 shows the results of the proposed approach with com-parison to the studies of Angel-Bello et al. [6].

Because the proposed approach uses the optimization techniqueon different machine block agent plans simultaneously, time to finda schedule takes so much time in some of the problem sets (suchas A034-4dm).

Fig. 17 shows the proposed approach Cmax solutions and bench-marks. As it can be seen obviously, the proposed approach followthe path of benchmark solutions where central heuristic is used tosolve the problem.

The proposed approach finds the scheduling results for tighterblocks faster. The results show that the proposed approach issuccessful for the manufacturing systems where maintenanceactivities are required frequently.

The AVRP set without considering aging capacities is a relaxedset of problem for the proposed approach because each machineblock agent is able to check the aging capacity (W) while checkingthe T constraint.

V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179 175

Fig. 15. Re-negotiation triggering mechanism after job cancelation event.

j2 j3 . . . jn-1

Machine Block Agent1 Schedule 1

j1 j4 jn

jk+1 jk+2 . . . jk+njk jk+2 jk+n

Machine Block Agent1 Schedule 2

jk+1 jk+2 . . . jk+njk j3 j4

j2 . . . jn-1j1 jn

hedul

6u

aPabnAl

TR

t

Machine Block Agent2 Schedule 1

Fig. 16. Machine block agent sc

.2. Part I – computational experiments with consideringtilization capacity

As stated previously, the proposed appoach is able to checkging capacity usage of jobs arriving to manufacturing system.roblem set in Section 6.1 is also used in this part of the study. Inddition to processing time, aging capacities are also consideredy using the load capacities of the same AVRP set. The mainte-

ance activity is selected as the first node in the problem sets A034,039, and A045. The weight given for the fist nodes are used as

ast node weight during testing. Fig. 18 shows the Cmax values for

able 2esults of computational experiments without considering maximum aging capacity.

CPLEX Benchmarkb

Obj. val. Time(s) Obj. val.

A034-2.25dm 1341 t.l.e. 1340

A034-2.5dm 1281a 4804.7 1281a

A034-3dm 1284a 3715.5 1295

A034-4dm 1245a 606.19 1245a

A039-2.25dm 1762 t.l.e. 1659

A039-2.5dm 1508 t.l.e. 1566

A039-3dm 1495a 10035 1537

A039-4dm 1521 t.l.e. 1578

A045-2.25dm 2617 t.l.e. 1764

A045-2.5dm 2530 t.l.e. 1825

A045-3dm 1576a 11160.3 1658

A045-4dm 1596 t.l.e. 1596

.l.e: Time limit exceeded.a Opt. solutions.b Solution found by Angel-Bello et al. [6], Gap (%): Percentage gap between heuristic anc

= 0.998; Tin = 397,996; Tf = 115.81; elmax = 5,000; � = 15; = 30; � = 0.3 s; � = 0.1 s.

Machine Block Agent2 Schedule 2

e change after re-negotiations.

different machine block size groups. As block sizes increase Cmax

values decrease proportionally.For example the proposed approach finds the best Cmax as 3140

for A034-2.25dm problem set with T = 747 and W = 350 ( = 0.998;Tin = 397,996; Tf = 115.86; elmax = 5000; � = 4; = 10, � = 1 s and� = 0.1 s). The schedule of the solution is given in Table 3.

As it can be seen from Table 3, the number of machine blockagents used for processing job agents is increased dramatically due

to aging capacity consideration. Table 4 shows machine schedul-ing results for different capacity levels and for different parametersets. Capacity levels are determined by the rate of T. Table 5 shows

Proposed approachc

Time(s) Obj. val. Time(s) Gap (%)

3.219 1340 2.517 0.03.243 1381 4.255 7.82.015 1360 5.758 5.01.031 1259 25.407 1.13.25 1787 2.467 7.74.062 1686 3.589 7.74.125 1565 3.745 1.84.422 1598 10.652 1.34.356 1926 1.115 9.24.675 1897 4.256 3.94.806 1813 8.138 9.34.931 1776 17.650 11.3

d model solutions.

176 V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179

Fig. 17. Benchmark of the proposed model with the static solutions of Angel-Bello et al. [6].

Fig. 18. Relationship between number of jobs and completion times.

Table 3Machine block agent job sequence.

Aging capacity Completion time (s)

Block 1: 0-28-14-32-0 217 446Block 2: 0-29-22-21-19-11-4-0 344 694Block 3: 0-24-20-31-13-2-0 346 470

rTc

f(

Table 5T and W values under aging-capacity.

2.25dm 2.5dm 3dm 4dm

TP

Block 4: 0-3-33-30-6-8-10-0 349 522Block 5: 0-15-12-18-9-5-0 336 731Block 6: 0-16-1-25-23-17-7 348 277

elated T and W values for different dm values. The ratio between values for different dm values is used to generate W values for

apacitated case.

Table 4 shows scheduling solutions of the proposed approach forour different parameter sets. One is determined as the base case

= 0.998; Tin = 397,996; Tf = 115.86; elmax = 5000; � = 4; = 10;

able 4roposed approach under aging-capacity (Cmax is in terms of seconds).

Problem set T W Obj. val.a Time(s)a Ob

A034Set1

A034-2.25dm 747 350 3140 2.782 28A034-2.5dm 830 388 3027 2.452 28A034-3dm 996 466 2589 2.139 23A034-4dm 1328 622 2239 2.235 22

A039Set2

A039-2.25dm 747 350 3628 6.020 36A039-2.5dm 830 388 3517 5.854 34A039-3dm 996 466 3197 6.002 31A039-4dm 1328 622 2939 4.661 28

A045Set3

A045-2.25dm 747 350 4029 3.279 39A045-2.5dm 830 388 3682 8.211 36A045-3dm 996 466 3333 3.593 33A045-4dm 1328 622 3235 3.949 32

a = 0.998; Tin = 397,996; Tf = 115.86; elmax = 5000; � = 4; = 10, � = 1 s, � = 0.1 s. a: base

b = 0.998; Tin = 397,996; Tf = 115.86; elmax = 5000; � = 6; = 10, � = 1 s, � = 0.1 s. b: case

c = 0.998; Tin = 397,996; Tf = 115.86; elmax = 5000; � = 4; = 10, � = 2 s, � = 0.1 s. c: case

d = 0.993; Tin = 77,983; Tf = 115.86; elmax = 927; � = 4; = 10, � = 1 s, � = 0.1 s. d: case 3.

T 747 830 996 1328W 350 388 466 622

� = 1 s; � = 0.1) and three other cases are determined to show thecapability of the proposed approach.

When base case and other cases are compared (case 1, case 2 andcase 3) the effect of parameter set on solution quality and time couldbe figured out. Objective function and computational time values

for base case and case 1 are given in Table 4. Parameter � value isdifferent in these two parameter sets. Parameter � is affecting opti-mization plan of machine block agent. Solution quality and timeto find solution increase proportionally when threshold value of

j. val.b Timeb Obj. val.c Timec Obj. val.d Timed

19 32.362 3184 1.612 3215 1.82753 31.325 3181 1.985 3166 1.68584 26.256 2785 1.120 2921 1.52004 37.286 2340 2.350 2676 1.925

01 85.625 3877 4.508 3864 3.93679 68.235 3888 3.235 3794 3.10861 63.925 3161 3.555 3369 3.20064 98.369 3234 2.436 3291 2.826

76 38.256 4197 2.586 4194 2.82570 111.520 3670 4.142 3823 4.42333 52.140 3333 2.652 3527 3.25222 85.652 3518 2.742 3427 2.687

case. 1.

2.

V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179 177

Fig. 19. Relationship between the base case and other cases objective function values.

e case

ocit˛es

oAg

6m

stF

Fig. 20. Relationship between the bas

ptimization is increased. Parameter � is different in base case andase 2. The parameter � is affecting the behaviors of the job agentsn the proposed approach. When the � value is increased the solu-ion quality and time to find solution are decreased proportionally.

and elmax parameters which are used in SA algorithm are differ-nt in base case and case 3. When and elmax are decreased theolution quality decreased proportionally.

Fig. 19 shows objective function values for the base case andther cases. Fig. 20 shows the computation times for the cases.lthough there are improvements in the Cmax values for case 1 asiven in Fig. 19, computational time increases dramatically.

.3. Part II – testing the proposed approach under dynamicanufacturing environment

In this part of the study full expressiveness of the dynamicystem is used to show the dynamic scheduling capability ofhe proposed approach. A graphical user interface (GUI) given inig. 21 is developed for the proposed multi-agent based dynamic

Fig. 21. Multi-agent based dynamic job control GUI.

and other cases computation times.

approach. A new job arrival could be scheduled in real time by usingthis interface.

Initially, the jobs in problem set A034-3dm are released to sys-tem randomly. Parameter set = 0.998; Tin = 397,996; Tf = 115.86;elmax = 5000; � = 4; = 10; � = 1 s; � = 0.1 s is used for finding theinitial solution. The jobs in this problem set are scheduled by theproposed approach. The schedule of jobs between maintenanceactivities is given in Table 6. Machine starts to process job 13 inthe first block after finishing initial scheduling. After obtaining theinitial schedule, a new job is released to the system by using the“New job arrival” menu of GUI in real time. When a new job entersto system, GUI asks to enter sij values for newly arriving job. Theuser of the program could also provide cij in advance. Table 7 showscij values for newly arriving job which are generated randomly forthis experiment.

When new job agent enters to system it negotiates with avail-able machine block agents. It autonomously finds a block to suitafter negotiation. Therefore there is no need to re-schedule allthe jobs in the system. Table 8 shows the schedule after theassignment of 34th job agent to manufacturing system. The sched-ule of the machine block agents are updated, this is because jobagent–machine agent interactions. Assignment of 34th job agent tosystem took only 130 ms. As it can be seen in both Tables 6 and 8,job 13 is the first job to be processed in machine block 1. If a job

agent starts to be processed then it autonomously prevents nego-tiations (see Fig. 14). Therefore, the schedule of job 13 does notchange during execution time.

Table 6Machine block agent job sequence (Cmax: 2633 s).

Agingcapacity

Completiontime (s)

Block 1: 0-13-9-7-6-27-29-1-3-0 439 578Block 2: 0-23-19-11-8-5-4-12-15-16-0 464 688Block 3: 0-32-10-20-22-28-2-0 451 619Block 4: 0-33-30-26-21-31-18-17-14-0 458 537Block 5: 0-25-24-0 128 211

178 V. Kaplanoglu / Applied Soft Computing 23 (2014) 165–179

Table 7Setup times for the newly arriving job.

From-To cij (s) From-To cij (s) From-To cij (s) From-To cij (s)

0-34 26 18-34 133 34-0 81 34-18 2241-34 28 19-34 126 34-1 95 34-19 1752-34 150 20-34 125 34-2 38 34-20 1653-34 125 21-34 17 34-3 54 34-21 1844-34 250 22-34 26 34-4 91 34-22 1295-34 125 23-34 29 34-5 138 34-23 1396-34 46 24-34 37 34-6 122 34-24 1197-34 20 25-34 26 34-7 145 34-25 1118-34 25 26-34 41 34-8 193 34-26 989-34 36 27-34 25 34-9 123 34-27 15110-34 88 28-34 100 34-10 170 34-28 12011-34 81 29-34 95 34-11 206 34-29 8312-34 131 30-34 84 34-12 119 34-30 2713-34 165 31-34 24 34-13 94 34-31 24314-34 121 32-34 33 34-14 119 34-32 14315-34 16 33-34 26

16-34 25 34-34 –

17-34 33

Table 8Machine block agent job sequence (Cmax: 2565 s).

Agingcapacity

Completiontime (s)

Block 1: 0-13-14-23-26-1-30-5-2-0 456 624Block 2: 0-22-17-10-32-4-6-3-0 456 593Block 3: 0-16-25-20-21-19-18-9-7-8-0 443 508Block 4: 0-12-11-31-27-28-29-33-0 462 640Block 5: 0-15-24-34-0 160 200

Table 9Machine block agent job sequence (Cmax: 2517 s).

Agingcapacity

Completiontime (s)

Block 1: 0-13-9-32-7-14-2-0 412 330Block 2: 0-10-12-25-24-19-22-26-27-34-0 466 571Block 3: 0-21-11-8-4-6-5-30-28-3-0 464 890

acbm1

7

sAdwcnbdatp

solp

[

[

[

[

[

Block 4: 0-16-1-33-20-31-18-17-0 454 508Block 5: 0-23-29-0 128 218

In addition to creating new job agents, a job could be canceled by system user while the system is operating. Job 15 is assumed to beanceled by the system user. When job 15 is canceled then machinelock schedules are re-arranged accordingly. Table 9 shows theachine schedule after job 15 is canceled. As given in Table 9, block

continues to process job 13 as the first operation.

. Concluding remarks and future works

Single machine scheduling problem with sequence-dependentetup times and availability constraints is considered in this paper.

multi-agent based optimization approach is proposed to han-le such kind of problem in a dynamic manufacturing systemhere single machine is scheduled. The proposed approach has the

apability to solve the problem with regular and irregular mainte-ance activities and aging capacity consideration. Jobs and machinelocks are designed as autonomous agents which make their ownecisions. Each machine block agent considers its self-optimalitynd negotiates with the job agents, respectively. Job agents tryo minimize the effect values to contribute to general schedulingerformance.

The proposed approach has the ability of both solving machine

cheduling problem under static and dynamic cases. In the casef static scheduling problem, proposed approach does not chal-enge with the full expressiveness of the dynamic system. Theerformance of proposed approach for static problems could be

[

[

34-15 112 34-33 3634-16 103 34-34 –34-17 184

dominated by other central static algorithms. The advantages ofthe proposed system are its ability to response newly arriving jobrequests and handling unexpected events in a very short time.When any job request is received by the system, it directly fit toa proper time block. The proposed approach could be adapted tothe real-time manufacturing environments with its distributed andself-reactive mechanism. The approach proposed has negotiationmechanism that reduces computational load of central optimiza-tion without guaranteeing the optimality.

The job priorities and job due date attributes could be articulatedto proposed approach as a future work. Some of the newly comingjobs might be given high priorities and can be finished due a specifictime according to manufacturing company policies.

References

[1] Y.M. Chen, S.C. Wang, Framework of agent-based intelligence system with two-stage decision-making process for distributed dynamic scheduling, Appl. SoftComput. 7 (2007) 229–245.

[2] R. Sarker, M. Omar, S.M.K. Hasan, D. Essam, Hybrid evolutionary algorithmfor job scheduling under machine maintenance, Appl. Soft Comput. 13 (2013)1440–1447.

[3] Z.L. Lin, Y.S. Huang, Nonperiodic preventive maintenance for repairable sys-tems, Naval Res. Logist. 57 (2010) 615–625.

[4] A. Duffua, O. Salih, A. Raouf, J. Campbell, Planning and Control of MaintenanceSystems, Modeling and Analysis, John Wiley & Sons, Inc., United States of Amer-ica, 1999.

[5] D. Ouelhadj, S. Petrovic, A survey of dynamic scheduling in manufacturingsystems, J. Sched. 12 (2009) 417–431.

[6] F. Angel-Bello, A. Alvarez, J. Pacheco, I. Martinez, A heuristic approach fora scheduling problem with periodic maintenance and sequence-dependentsetup times, Comput. Math. Appl. 61 (2011) 797–808.

[7] M.A. Gonzalez, Metaheuristic solutions to the “Job shop scheduling problemwith sequence-dependent setup times”, AI Commun. 26 (2013) 419–421.

[8] Z.X. Guo, W.K. Wong, S.Y.S. Leung, J.T. Fan, S.F. Chan, Genetic optimizationof order scheduling with multiple uncertainties, Expert Syst. Appl. 35 (2008)1788–1801.

[9] C.Y. Lee, S.D. Liman, Single-machine flow-time scheduling with scheduledmaintenance, Acta Inf. 29 (1992) 375–382.

10] S.J. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall,Upper Saddle River, NJ, 2010.

11] C. Koulamas, G.J. Kyparisis, Single-machine scheduling problems with past-sequence-dependent delivery times, Int. J. Prod. Econ. 126 (2010) 264–266.

12] Y.Q. Yin, M. Liu, T.C.E. Cheng, C.C. Wu, S.R. Cheng, Four single-machine schedul-ing problems involving due date determination decisions, Inf. Sci. 251 (2013)164–181.

13] J.S. Chen, Scheduling of nonresumable jobs and flexible maintenance activitieson a single machine to minimize makespan, Eur. J. Oper. Res. 190 (2008) 90–102.

14] H.C. Hwang, S.Y. Chang, Parallel machines scheduling with machine shutdowns,

Comput. Math. Appl. 36 (1998) 21–31.

15] G. Schmidt, Scheduling with limited machine availability, Eur. J. Oper. Res. 121(2000) 1–15.

16] H.C. Hwang, K. Lee, S.Y. Chang, The effect of machine availability on the worst-case performance of LPT, Discrete Appl. Math. 148 (2005) 49–61.

t Com

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[46] W. Bennage, A. Dhingra, Single and multiobjective structural optimization in

V. Kaplanoglu / Applied Sof

17] C. Low, C.J. Hsu, C.T. Su, Minimizing the makespan with an availability con-straint on a single machine under simple linear deterioration, Comput. Math.Appl. 56 (2008) 257–265.

18] G. Mosheiov, A. Sarig, Scheduling a maintenance activity to minimize totalweighted completion-time, Comput. Math. Appl. 57 (2009) 619–623.

19] V.M. Dalfard, G. Mohammadi, Two meta-heuristic algorithms for solving multi-objective flexible job-shop scheduling with parallel machine and maintenanceconstraints, Comput. Math. Appl. 64 (2012) 2111–2117.

20] S.L. Khelifati, F. Benbouzid-Sitayeb, A multi-agent scheduling approach for thejoint scheduling of jobs and maintenance operations in the flow shop sequenc-ing problem, Comput. Collect. Intell. Technol. Appl. 6923 (2011) 60–69.

21] L.Z. Qin, Q.S. Li, ILOG and MAS based framework of job-shop scheduling system,Adv. Res. Comput. Educ. Simul. Model. Pt I 175 (2011) 238–242.

22] L. Duan, M.K. Dogru, U. Ozen, J.C. Beck, A negotiation framework for linked com-binatorial optimization problems, Auton. Agents Multi-Agent Syst. 25 (2012)158–182.

23] R. Erol, C. Sahin, A. Baykasoglu, V. Kaplanoglu, A multi-agent based approachto dynamic scheduling of machines and automated guided vehicles in manu-facturing systems, Appl. Soft Comput. 12 (2012) 1720–1732.

24] P. Lou, Q. Liu, Z.D. Zhou, H.Q. Wang, S.X. Sun, Multi-agent-based proactive-reactive scheduling for a job shop, Int. J. Adv. Manuf. Technol. 59 (2012)311–324.

25] L.M. Hercog, Better manufacturing process organization using multi-agent self-organization and co-evolutionary classifier systems: the multibar problem,Appl. Soft Comput. 13 (2013) 1407–1418.

26] S. Polyakovskiy, R. M’Hallah, A multi-agent system for the weighted earlinesstardiness parallel machine problem, Comput. Oper. Res. 44 (2014) 115–136.

27] A. Henchiri, M. Ennigrou, Particle swarm optimization combined with tabusearch in a multi-agent model for flexible job shop problem, Lect. Notes Com-put. Sci. 7929 (2013) 385–394.

28] G. Wang, Q. Chen, Multi-agent single machine scheduling with controllableprocessing times, in: Jisuanji Jicheng Zhizao Xitong/Computer Integrated Man-ufacturing Systems CIMS 2013, 2013, pp. 7078–7082.

29] C. Grimme, J. Lepping, U. Schwiegelshohn, Multi-criteria scheduling: an agent-based approach for expert knowledge integration, J. Sched. 16 (2013) 369–383.

30] M.M. Savino, A. Mazza, G. Neubert, Agent-based flow-shop modelling indynamic environment, Prod. Plann. Control 25 (2014) 110–122.

31] M. Merdan, T. Moser, W. Sunindyo, S. Biffl, P. Vrba, Workflow scheduling usingmulti-agent systems in a dynamically changing environment, J. Simul. 7 (2013)144–158.

[

puting 23 (2014) 165–179 179

32] C.J. Huang, L.M. Liao, A multi-agent-based negotiation approach for parallelmachine scheduling with multi-objectives in an electro-etching process, Int. J.Prod. Res. 50 (2012) 5719–5733.

33] C. Hahn, C. Madrigal-Mora, K. Fischer, A platform-independent metamodel formultiagent systems, Auton. Agents Multi-Agent Syst. 18 (2009) 239–266.

34] Y.M. Chen, B.Y. Wang, A study on modeling of human spatial behavior usingmulti-agent technique, Expert Syst. Appl. 39 (2012) 3048–3060.

35] Y.M. Chen, W.-Y. Wu, Cooperative electronic attack for groups of unmannedair vehicles based on multi-agent simulation and evaluation, Int. J. Comput.Sci. Issues 9 (2012) 11–16.

36] Y.M. Chen, H.-M. Yeh, An implementation of the multiagent system for market-based cloud resource allocation, J. Comput. 2 (2010) 27–33.

37] A. Rajabinasab, S. Mansour, Dynamic flexible job shop scheduling with alter-native process plans: an agent-based approach, Int. J. Adv. Manuf. Technol. 54(2011) 1091–1107.

38] I. Cil, M. Mala, A multi-agent architecture for modelling and simulation ofsmall military unit combat in asymmetric warfare, Expert Syst. Appl. 37 (2010)1331–1343.

39] V.R. Komma, P.K. Jain, N.K. Mehta, An approach for agent modeling in manufac-turing on JADETM reactive architecture, Int. J. Adv. Manuf. Technol. 52 (2011)1079–1090.

40] AOS, JACK Intelligent Agents® Agent Manual, Agent Oriented Software Pty., Ltd.,Carlton South, Victoria, Australia, 2009.

41] W.L. Winston, Operations Research, Applications and Algorithms, 3rd ed., Indi-ana University, USA, 1994.

42] S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimization by simulated annealing,Science 220 (1983) 671–680.

43] W.J. Xia, Z.M. Wu, An effective hybrid optimization approach for multi-objective flexible job-shop scheduling problems, Comput. Ind. Eng. 48 (2005)409–425.

44] P.J.M. Vanlaarhoven, E.H.L. Aarts, J.K. Lenstra, Job shop scheduling by simulatedannealing, Oper. Res. 40 (1992) 113–125.

45] A. Baykasoglu, N.N.Z. Gindy, A simulated annealing algorithm for dynamic lay-out problem, Comput. Oper. Res. 28 (2001) 1403–1426.

discrete–continuous variables using simulated annealing, Int. J. Numer. Meth-ods Eng. 38 (1995) 2753–2773.

47] D. Vigo, http://www.or.deis.unibo.it/research pages/ORinstances/VRPLIB/VRPLIB.html (last-access-date: 16.11.13).