Towards the Application of Case BasedReasoning to Decision-Making in Concurrent

Product Development (Concurrent Engineering).

B. U. Haque, R. A. Belecheanu, R. J. Barson, K. S. PawarSchool of Mechanical, Materials, Manufacturing Engineering and

Management, University of Nottingham,Nottingham, United Kingdom.

Abstract

This paper describes the development and application of CaseBased Reasoning (CBR) to provide decision support forproject managers and engineers during the early phases ofNew Product Development (NPD) in a ConcurrentEngineering (CE) environment. The paper discusses thereasons for using CBR, focussing on issues such as casecollection, maintenance, terminology, adaptation, andsimilarity; and how the final system could contribute towardsachieving a CE conducive culture. The main issues in usingCBR in a CE environment, that is characterised by ill definedand ill structured information during early phases of productdevelopment, are textual consistency of terminology, validityof case similarity, and the difficulty in automating caseevaluation and adaptation. Additionally the paper concludesthat using technology like CBR, which can be costly todevelop and implement, requires the company to trainconsiderably their managers and team members to documenttheir experiences and knowledge in a manner with which thesystem can work with and team members can understand.There needs to a commitment to maintain and improve theknowledge base- a ‘knowledge friendly’ culture hence needsto be instilled for CBR tools to succeed.

1 Problem Description

1.1 Background (Concurrent Engineering)

New Product Development (NPD), is an interdisciplinary activity requiringcontributions from nearly all the functions of a firm, whether it is anupgrade/improvement of an existing product or a new concept either to the companyor the market. Traditionally NPD has been viewed as an organisational activity,which was the result of various functional activities performed in stages from

concept development to product delivery. The sequential operation of thesefunctional stages resulted in long development times and many quality problems dueto the lack of communication and understanding of the different product design,manufacturing and above all customer requirements. To avoid these problemsConcurrent Engineering (CE) is being used by many companies and has resulted incompanies making new products better and faster [1, 2, 3, 4].

CE or CNPD is characterised by the early involvement of the different functionaldisciplines and parallelism of hitherto sequential activities (i.e., bringingdownstream activities forward). Decisions made in the early (design) phases ofproduct development are hence often based on incomplete, ill-structured, and poorquality information. This is why decisions are sometimes made in an empiricalmanner, using only personal knowledge and experience, gained during past problemsolving processes. It is widely cited that most mangers/designers refer back toprevious solutions to related problems as a first step in the design process [5].

1.2 The Problem (End User Needs)

When engineers or managers call upon past experience or the “experts” opinion, theinformation or knowledge is prone to bias of the particular experiences of thatindividual (or the so-called “expert”). The wider collective company (corporate)knowledge is not always readily available in a structured and consistent format. Adetailed review of project management procedures and processes in new productdevelopment at selected companies [6] identified that there exists a case history ofpast projects contained in disparate ‘data’ or ‘information’ sources such as projectfiles, databases and most importantly individuals memory. This information was notonly restricted to major decisions concerning the continuation of the project, but alsoincluded data about specific products or components, design decisions that haveworked well in the past and those that have not worked so well, problem solvingapproaches etc. This data was usually difficult to access, especially where individualknowledge or memory was concerned, when making decisions in new projects. Atthe same time it was not always possible to find the most experienced or mostknowledgeable personnel. Hence, there was the risk that a problem or difficulty thatwas found in an earlier project, and subsequently resolved, could be repeated in anew project.

There was a lack of structured support, not only in formal management reviews(decisions), but also in many decisions made by team members outside the formalreviews with respect to the detail design and development. These decisions could beequally critical to the success of the project. The basic requirement of the industrialpartners was that the past experience was presented in a constructive way at the timeof making the decision, and also indicate the relevance of the data for that particulardecision. The decision support data or knowledge should also be examined by otherteam members to assess from their viewpoint the acceptability of the decision. Therequirement for the system was hence to build a knowledge base in which complete‘decision cases’ or scenarios could be entered and then recalled or reused whensimilar problems arose again.

Another issue was that design is typically carried out in an iterative manner in termsof generating the initial design and then testing. Product or component design, inmechanical engineering for instance, is evaluated under numerous interrelatedcriteria such as machinability, quality, reliability, structural integrity, assembly,maintenance and so on. This process is referred to as Design For x (DFx), with x asone of the criteria or constraints. Time delays and costs can be incurred if suchevaluations result in red-design or take too long. Though the CE philosophyattempts to bring DFx issues to the attention of the designer as soon as possible, theprocess would benefit greatly if support could be provided through a what-if studybased on past experiences.

Design changes or changes to specification arise from other sources too such asmarketing, reacting to sudden changes in market needs or issues relating toindustrial design; or purchasing, identifying supplier capability limitations, etc.Quite often engineers are not aware of the consequences of the changes. It would bequite useful if the consequences of such changes or similar changes could beidentified or known in advance or prior to elaborate testing, simulations or waitingfor the actual event to happen!

The above problems or issues called for a knowledge based decision support system,providing the managers and engineers (in design and development) with structured,consistent, comprehensive and accurate information and knowledge. This wouldenable the early phases of NPD and hence CE to be more productive. Additionallythe success of CE depends upon collaboration between the different functionalexpertise to arrive at a mutually agreeable decision. The decision support shouldencourage this by providing viewpoints from different experiences of differentpeople.

The development of the required system has been carried out in an EU fundedproject called CODESCO- A Practical Communication and Decision SupportEnvironment for Managing Concurrent Product Development (ESPRIT project no.25455) [7]. The overall objective of this research project was to develop and validatea communication as well as a decision support system for helping project managersand design/development engineers in their decision-making activities within a CEenvironment.

2 Application Description

2.1 The Solution- Choice of CBR

The explicit request for reuse of knowledge and experience called for the applicationof Case Based Reasoning (CBR). CBR is a computer technique, which combines theknowledge based support philosophy with a simulation of human reasoning whenpast experience is used, i.e. mentally searching for similar situations happened in thepast and reusing the experience gained in those situations [8]. In the same way, inCBR, the knowledge cases are structured and stored in a database, which the userqueries when trying to solve a problem. The system retrieves a set of similar cases

and then evaluates the similarity between each case in the database and the query.The most similar case(s) are presented to the user as possible scenarios for theproblem at hand. The user has to decide if the solution retrieved is applicable to theproblem, i.e. the system doesn’t make the decision, it only supports the decisionmaking process. If it cannot be reused, the solution is adapted (manually orautomatically). When the user finds a solution, and its validity has been determined,it is retained with the problem as a new case in the database (the case is “learned”),for future reuse.

The theoretical CBR cycle is, therefore, a retrieve-evaluate-adapt-learn process.However, a CBR system may very well implement only the retrieval stage of theprocess and does not need to implement the other stages. The retrieval stage is thebasic stage and the expression of the concept of reuse of experience.

There were a number of reasons for choosing CBR over conventional rule basedsystems. The main requirement for the system was that it should be able to support avariety of product and business domains. Additionally the system was intended todeal with a fairly wide range of technical and managerial problems. Traditional rule-based knowledge approaches were not found suitable for this requirement, as theyrequired strong domain knowledge and representation, whereas decision problems inCE are generally difficult to define and structure. In CBR, as opposed to rule-basedapproaches, knowledge about the domain is acquired and maintained throughunrelated but similar cases and does not need a domain expert or knowledge aboutthe problem domain. The generic concept of our system hence moves away from arule-based implementation, as for each type of product and company context,specific rule-sets would need to be encoded.

Another requirement was that the system should be able to trace the effect ofdecisions made in upstream process on the down stream processes, i.e., perform aWhat-if Analysis. CBR enables this to be performed in a better way using the samesearching mechanism and case structure as it would for a normal CBR query. Thisenables identification of different effects or consequences of change for differentcontexts and conditions without having the need to build a complex set of rules fordifferent contexts and conditions. Additionally, a more detailed analysis can beperformed, through sequential search, and one can carry on the search if theretrieved consequences are not acceptable after one iteration.

2.2 Collection of Decision Cases and Development of the CaseStructure

One of the main development issues was the definition of the case structure, as itsacceptance to both the end user and the system developer was paramount to thecontinuation of the project. In the CODECSO project two manufacturingorganisations are taking part in the development, implementation and validation ofthe CBR system. The two industrial partners are:

1. Thomson CSF Service Industrie (TSI). TSI are located in France and producehigh technology electrical and electronic hardware, and information technologysystems (software), for defence and commercial applications. Their products arequite often ‘one of a kind’.

2. General Domestic Appliances Ltd. (GDA). GDA, are a subsidiary company of ajoint venture between industrial giants GE (USA) and GEC (UK). GDA Ltd.,are a consumer goods company, producing domestic appliances in the UK.

Analysis of decision making literature [9], CBR literature [10] and mapping of realdecision making activity during new product development at the industrial userslead to the development of a generic decision making process. Essentially fourphases in decision making were identified;

(1) Framing or problem understanding;(2) Gathering intelligence and developing alternatives or solutions;(3) Coming to conclusions and selecting a solution; and(4) Learning from feedback.

These phases were used to define the structure of decision cases to be collected andentered into a knowledge base, for re-use during new product developmentactivities.

Learning from feedback or experience

Framing or Problem Understanding

Identification ofthe problem (issue)

Is the contextknown ?

Examination of thecontext

Assessment of theproblem

Gathering Intelligence / GeneratingAlternatives / Developing solutions

Creation/developmentof the solutions

(alternatives)

Assessment of thesolutions- advantages/

disadvantages

Are solutions validalternatives?

Coming to conclusions/ Solution Selection

Selection of solutionusing selection

criteria

Implementation ofselected solution

Outcomeidentification

NO

YES

YES

NO

CODESCOCase Case

Enter Decsion Caseinto Case Base

Learning from CaseBase

Figure 1 The Generic Decision Making Process

In Figure 1 a model of the generic decision process is presented. Further, thedifferent phases of the decision process are named. For the actual case structure tobe used in the CBR application this process was simplified into three phases:

• Problem Description• Solution Development• Outcome

The Problem Description section, besides a textual description of the problem,incorporates details about the context in which the problem arose: product, projectand development phase information, people involved or responsible for the decision-making, and decision parameters. The Solution Development section aims to recordhow the solution was found, what the available alternatives were, why one solutionwas preferred to the others. It was also found useful to record the Outcome ofimplementing that solution, in terms of performance, and positive or negativeconsequences. In this way, retrieving cases was also useful for identifying andanalysing ‘what-if’ scenarios related to particular problems.

Having defined the case structure further interviews, using the predefined casestructures, were carried out to collect decision cases in both companies. Collectionand analysis of decision cases lead to the identification of different types ofdecisions made in new product development. Essentially two main types wereidentified, managerial/strategic decisions or technical/design decisions. These typeswere further subdivided. Under managerial/strategic decisions the main groupingswere: (1) Determining the risks of the project; (2) Cost Estimations; (3) Teambuilding decisions; (4) Determining strategic indicators for the project; and (5)Choosing between strategic alternatives e.g., make versus buy decisions. Undertechnical decisions the main groupings were: (1) Choosing between different designalternatives or technical alternatives relating to production etc.; and (2) Identifyingtechnical risks.

The differences between the operating (trading) and cultural environments of thetwo companies meant that at a detailed level the case structure had to be customisedto reflect the differences. For instance, in TSI the product is fairly complex (e.g.rugged computers, radars, and avionics display systems), the business is very muchgeared towards satisfying individual client needs and the contracts quite ofteninclude long-term maintenance and life-cycle support. So, ‘client information’ isincluded only in TSI cases. Additionally because of their business environment, thefocus of TSI decisions were on risk analysis, cost estimation, determining strategicindicators for the project, and choosing between strategic alternatives. At GDA, withthe products made for a mass market the focus was more on marketing, quality andtechnical issues, like choosing between different design or technical alternatives, andidentifying technical risks.

PROBLEM

SOLUTION DEVELOPMENT AND SOLUTION SELECTION

OUTCOME

Problem/Situation Description

Solution

Outcome

Objectives

Alternatives

Criteria

Consequences

• Problem description

• Context

• Constraints

• Possible solutions:

Selected solution

- advantages

- disadvantages

• Selection criteria

• Implemented solution

• Consequences

• Comments

PROBLEM SOLVING ISSUES

CASE STRUCTURE DECISION MAKING ISSUES

- solution

• Method of assessment

Figure 2 Details of the Case Structure

Also, the general decision parameters, referred in the cases as ‘contractual orcompany constraints’, have been found to be different, i.e. decision-makers considerdifferent aspects in making a decision. Cost to the client, delivery time, technicalperformance and technology are specifically akin to TSI, whereas quality and safetystandards, company costs, marketing (such as aesthetics) and production issues(such as tooling) are the major decision constraints in GDA. Nevertheless, thegeneral process of making decisions was the same, independent of function,expertise, or type of company, and this is reflected in the common high levelstructure of the case.

Figure 2 shows the main attributes or fields, common to GDA and TSI. At this levelof detail, cases from both companies can be mapped onto this structure. Differences,as mentioned above, appear in the sub fields of fields like ‘context’ and‘constraints’. For example ‘Context’ contains sub fields like ‘product details’,‘project details’, which obviously differ from company to company.

The fields and hence the case structure (and its specialisation for GDA and TSI)developed can be used for a wide range of companies similar in structure, product orNPD process characteristics. The NPD processes and management structure foundin TSI and GDA are common to many companies of similar business or producttype. This is mainly due to that fact that most companies’ quality assuranceprocedures are based on ISO9000 standards, and generic best practice procedures.

2.3 System Architecture and CBR Tool Selection

2.3.1 Architecture

Figure 3 illustrates the architecture of the Decision Support System within the widerCODESCO system. At User Interface Level, Java has been used for developing the‘forms’ for entering cases, case querying and query results. Distinct interfaces havebeen created for both GDA and TSI. At Decision Support Level, The Easy Reasoner(TER) CBR tool from Haley Enterprise, Inc. has been used. Using an existing CBRtool was appropriate for our project considering the time and effort available.

The Easy Resoner (TER)(C libraries)

CBR m odule(C application)

CBR Display

Decision Support M odule

User Interface Level

Decision Support Level

dBase

Database ofCases

Figure 3 Architecture of the Decision Support System

2.3.2 Selection of CBR Tool:

A review of the CBR tool market identfied sixteen (16) suitable tools. An initialscreening process using ten criteria (with constraints), see Table 1, reduced this listdown to four potential contenders that could satisfy most of our criteria/constraints.

Criteria ConstraintsOperating system platform Windows NT and UNIXCustomisability of API Java or C++Ability to represent cases collected to dateCosts Less then $3.000 USDReasoning speedNumber of cases it can handle A no limit situation is desiredThe reasoning mechanismTraining support for developers Training should be providedPost development release/version support Lower price for future s/w releasesHear say/comments Degree of confidence in source

Table 1 CBR Tool Selection Criteria and Constraints

The Easy Reasoner Kate CBR-Works ReCallThe Haley Enterprise Inc Aknosoft TecInno GmbH Isoft

Score Score Score Score

Ability to representcases

CODESCO Casestructure could beimplemented

5 Cannot represent Casestructure for CODESCOstructure. Kate cannotsupport adaptation

0 CASUEL syntax for caserepresentation

3 Full case representationproviding complex objecthierarchy

3

Reasoning Dimension & speed

C++ implementationincreases the speed

3 2 seconds to retrieve a caseout of 10.000 case-base on aPC

5 Retrieval in less than 1second, out ofapproximately 1000cases

3 Information Not Available 0

Customization C, C++ API 5 Also Available as DLLallowing integration in userapplications

5 Requires knowledge ofSMALLTALKprogramming

3 Libraries C/C++ calls forembedding your ReCallapplication into othersapplications

5

Database support Supports access toODBC and SQLdatabases

3 Open to databases (doesn’tspecify which ones) forimport/export capabilities

5 DB2, Oracle, Sybase,GemStone, ODBCcompatible RDBMS

5 Connection to RDBMS viaODBC driver

5

Platforms Win 3.1,95,NT,Unix 5 Win 3.1,95,NT, soon Unix 3 Win 3.1,95,NT,Unix,OS/2, Mac, Solaris

5 Win 3.1,95,NT,Unix 5

Price RETE++ 1850 ECUTER 4000 ECU

4 13875 ECU CommercialLicense; 2313 ECUAcademic License

1 2313 ECU 5 Information Not Available 0

Total Scored 25 19 24 18

Table 2 Comparison of Four CBR Tools

The four tools were evaluated against each other using six of the ten selectioncriteria described above, but this time applying a more rigorous scoring mechanism,see Table 2. TER was identified as the most suitable tool for our application. Thetool comes as a set of Eclipse and C libraries, which implement theindexing and retrieval mechanism. The Searching Engine is based onEclipse libraries (Clips-like language) but TER also provides the layer ofC libraries upon these Eclipse libraries. Unlike the other toolsresearched, it is not a ‘ready-to-use’ software tool, and therefore requiresmore application programming effort.

Summarising, the following advantages of TER were the basis for its selection:

• The tool can be embedded in a C++ application and therefore the input/outputinterface which our system requires can be built (enabling the customisabilityfeature)

• The system works with an external database; the current version works only withthe ‘dBase 4’ database format, but any other format can be converted to this one.Further releases will include ODBC functionality.

• The tool allows for text pattern search, which is a critical need for our system asthe cases contain almost only textual information.

Despite the fact that the case structure described earlier was done in a hierarchicalmanner, decomposition of fields into sub fields (or sections) can be seen only at theinterface level of the application. The representation used to store the cases in thedatabase is a flat representation (as opposed to hierarchical). Two databases withdifferent structures, have been created, one for TSI and one for GDA, each databaseconsisting of one table. The database format was dBase 4, as required by the CBRtool.

The fields in the database have been represented as text fields, hence, text similarityformulas have been used in the implementation of the CBR module, to calculate thesimilarity between cases. The underlying reason is the difficulty to further structuresome of the fields, like Problem Description, Solution, Constraints.

2.4 Functionality (including What if Study)

The following functions will be provided by the CBR system:

• Insert, modify, delete, and browse through cases in the database• The ability to customise the input or user interface according to the case structure

required by the user company• CBR-like query of the database- to search for a particular problem or solution.• What-if analysis- i.e., what would happen if a particular decision is taken.

The what if analysis is perhaps the most useful functionality as far as the end usersare concerned. Below is a description of how we intend to support this.

The ‘what-if study’ function: To capture design or specification change scenariosthe information and consequently knowledge needs to be presented in a way, whichwould enable retrieval. Therefore, in such scenarios, problems can be described interms of a ‘change’ (in specification or design) and a ‘consequence’ (or the‘outcome’).

Change in design or other parameter

Query database for effects of change

Results

with + ve consequences with -ve consequences

implement change Acceptable?

Search for solution in database

Solution found?Can’t implement change

Is solution adesign change?

Can implement changeunder new conditions

NO

NO

NO

YES

YES

ST OP

ST ART

Figure 4 What if Study

The CBR system can retrieve the consequence information by either searching(querying) the problem description field (which would contain the change and theoutcome or consequence information) or the solution implemented and outcomefields, if the change was itself a solution to another problem. The flow chart inFigure 4 summarises the ‘what-if’ procedure. The starting point is of course achange in design, or other specification or issue such as increased costs, faultycomponent, manufacturing problem etc. The results of the query would reveal eitherpositive or negative consequences, described either in the problem description fieldor the outcome field. If the consequences are positive then the change isimplementable, otherwise a solution to the problem or change needs to be found.The results of the second search (for a solution) could reveal that no solution isavailable, or a solution is available under new conditions, or that a further designchange is required, hence another what-if loop. The CBR system hence identifies

scenarios that have happened in the past, in similar contexts, and for similar designchanges.

Through a sequential querying of the case base, the user can see and analyse thedynamics of the design process, retracing upstream changes and finding downstreamconsequences. The scenario can continue depending on the availability of cases andon how positive the outcome is (negative consequences can be new problems to beinvestigated). This functionality would aid quite considerably the achievement of amore concurrent product development environment.

3 Application Building

3.1 The Development Process- overview

A CBR implementation, in any domain, requires a detailed analysis of theenvironment, as it is strongly related to the type of problems being solved ordecisions being supported. For this reason, in the CODESCO project, the academicpartners started by looking at the specific product development context of eachcompany. User requirements were developed into system functional requirementsthrough interviews, questionnaires and focussed workshops with project managers,design/development engineers, production, and quality engineers.

GDA

TSI

Industrialenvironment

Literature Research

CaseStructure

Development

Case collectionthrough interviews

Identification ofCBR tool requirements,

CBR system requirements

CBR tool selection

CBR system implementation

Testing in the usercompanies and getting

user feedback

Refinement

Case base development

More case collection

Refinement

Requirements Definition Case structure definitionand case collection

Implementation and pilot projects

User RequirementsSystem Requirements

Figure 5 Development Phases

The system was developed based on industrial requirements formulated by thecompanies participating as partners in CODESCO. Figure 5 above illustrates themain phases of development. The diagram shows three phases

Requirements definition: This involved collection of user requirements andestablishment of the system requirements.

Case structure definition and case collection: The application of CBR requires astructured representation of knowledge in the form of cases.

Implementation: This involved selecting the commercial CBR tool to be used fordevelopment of the application, and then customisation of the tool to meet bothcommon and specific end user requirements. Below we shall describe in detail thedevelopment of the CBR system.

3.2 CBR System Development Tasks

The main tasks of the development phase are:

1. To collect cases and build the database of cases2. To implement the TER application using the TER built-in functions (TER

Search Engine) for indexing and retrieval.3. To implement the input/output interface4. To implement the information retrieval for the ‘what-if’ analysis.

3.2.1 Database Development

In order to implement the first level of the development tasks, i.e., the database ofcases, over 40 cases from TSI and GDA were collected and entered in a dBase 4database. Both cases from TSI and GDA were entered using the same case templatecustomised to meet individual organisation needs, in separate files.

3.2.2 TER Application Development

The CBR Module (see Figure 3) is a C application, in which the query data providedby the user through the CBR Display module is processed and the results consist ofthe ‘n’ most similar cases (‘n’ is user defined), or cases with similarity less than auser specified threshold.

The implementation of the retrieval mechanism is provided with The Easy Reasoner,in the form of C libraries and consists of two main phases, a pre-query processingphase and a query-processing phase. In the first phase, an index containing statisticalinformation is created for all the records in the database, before queries are made. Inthe query processing stage, information contained in the index is used to determinethe case(s) most similar to the query, using a ‘Nearest Neighbour’ algorithm. Thesimilarity between two cases is calculated pairwise, between pairs of fields. Thefields in the database have been represented as free text fields; hence the similarityformulae for fields are textual retrieval specific. During calculation, each text field isconsidered a list of terms (words) and the information in the field is normalised, sothat each field contributes evenly to the global distance of two cases. Therefore, aweight is determined for each term in the text field of the case (record), and a weightis determined for each term in the text field of the query. These weights make part ofthe index. The weight of the k-th term in the record i, is

Fik Log(nk/N)

√Σj(Fij Log(nj/N))²Wik =

,where N is the number of records, nk is the number of different records in whichterm k occurs, and Fik is the number of occurrences of term k in record i divided bythe total number of terms in record i. Using these weights, a normalised distancebetween two fields is calculated according to the formula:

Σk WikWk

√Σk (Wik)² Σk (Wk) ² Si =

,where Wk - the weight of the k-th term in the query.

The similarities are calculated only for the fields with information. Therefore theresults depend on how detailed the query as well the cases are.

3.2.3 User Interface Development

The user input interface or CBR Display module is a set of windows (entry forms)implemented in Java, where a query can be created by entering data in thecorresponding fields (see Figure 6). A query contains data related to Problem,Context and Constraints. The results are presented as a list of similar cases, orderedby similarity, in a separate window, where details about the Solutions and how theyhave been developed can be viewed (see Figure 7). The CBR Display moduleinterfaces with the CBR module through a temporary file containing the query.

3.2.4 ‘What-If’ analysis implementation task

The knowledge structure was not only suitable for design problem solving task, butalso for the analysis of the ‘what-if’ scenarios. As problems are described in termsof ‘cause – effects’, cases can be retrieved in such a way, that consequences of adesign change can be identified, as well as the cause(s) for a particular problem.

Figure 6 Some Screen Shots from a CBR Query

Figure 7 Results of a CBR Query

3.3 The project team, costs and time scales

The project team, costs and time scales for each of the three phases identified aboveare described below (Table 3):

DevelopmentPhase

Team Duration(months)

Effort inMan

Months

TotalDevelopmentCosts (Euro)

RequirementsDefinition

Lead by the industrialpartners. Representativesfrom the two industrial endusers (project managers), 2academic institutes, and 2software houses/consultancies

6 8.3 89K

Case StructureDefinition,Case Collectionand Analysis

Lead by one academicinstitute with support from theother academic institute, andthe industrial end userrepresentative/s.

6 18.4 78K

Implementation-Development(excl. PilotProjects)

Lead by the software house,supported by the academicpartners. Essentially selecting,customising, and integratingthe commercial tool.

12 19.8 174K

TOTAL(OVERALL)

15 (due tooverlapping of tasks)

46.5 341K

Table 3 Project Team, Times Scales, Manpower, and Costs (in Euro)

The table shows that the pre-implementation tasks, i.e., requirements and casestructure definition and collection, account for roughly half of the total costs.

3.4 Installation and End User Training

A well-defined training procedure was required in order to ensure that the expectedresults were obtained when using the application, at the two test sites. Firstly, thesystem developers installed the software with the assistance of a computeradministrator from the company. A Computer Based Training package wasdeveloped using Macromedia (a multimedia authoring package) and individual setsof usage guidelines for GDA and TSI were written. The developers carried out thetraining sessions, and the user group was made of a Project Manager, a TechnicalManager and a design team member. A general presentation of the software wasmade at the beginning, followed by the CBT and the specific guidelines. At the end,a user was asked to identify a current or recent technical or managerial problem anduse the system to find a solution.

In the second stage, the user participants in the first training session were asked toperform training for other possible users in the company, using the CBT.

3.5 Current Status of Development

At the present a prototype has been implemented, i.e. a program able to retrievecases to match with the ‘Problem Situation/Description’. The results consist of a listof similar cases, in a decreasing order of similarity.

The program is now currently being tested at the industrial end user sites, ondifferent queries and with different input parameters for the similarity function. Theimplementation will be finalised after the prototype is thoroughly tested in thecompanies. Several issues will be considered during the testing phase:

• the performance of the system in terms of speed and accuracy of the results; thisentails looking also at the performance of the CBR tool, in terms of textualretrieval capabilities and any possible limitations of the database capacity

• the user feedback, regarding:- the relevance of the information retrieved (“how suitable is the solution

retrieved for the current problem ?”)- the relevance of the fields in the case structure- the ease-of use of the system, in terms of the entry forms layout and level of

detail.• ‘how to use the software’, in terms of relationship between the detail of data

entered in the query and the accuracy of the results.

Additionally during refinement, customisability of the interface and the way ofdefining the case structure will be approached.

3.6 Issues Emerging

The development has raised a number of important issues for people interested inusing case based reasoning for supporting decision making in NPD. These arediscussed below.

3.6.1 Case Collection and Maintenance

Case collection and the development of an appropriate structure has been animportant issue for the success of CBR implementation and hence the decisionsupport module. Training the end users on how to collect and enter cases isimportant. Additionally, the type of cases collected is very important. Cases thathave a short shelf life in terms of validity of knowledge (i.e., obsolete knowledgewithin a year or earlier) will not be included in the case base to guarantee a highrelevance for industrial use of the cases. The case base has to be maintained andupdated regularly. Once the user makes a decision, based on the informationprovided by the CBR system, the new solution has to be implemented (tested) inorder to find the outcome. This solution and its outcome will be “retained” togetherwith the problem (query) as a new case in the database.

3.6.2 Case Terminology

To enforce consistency of data across a case base, the terminology used indescribing the cases has to be specific for that company. Users must use the sameterminology for the same concepts (e.g. the NPD phases are usually formalised). Alack of consistent terminology could lead to problems with case matching for thecase similarity function- the most relevant cases could be missed due to text basedsimilarity calculations.

In order to help this, a taxonomy (list of values for a field) is suggested by thesystem, where possible. For instance the ‘Program Main Phase’ field in TSI casescan take the following values: Design, Production or Maintenance. However, thesevalues are a “guideline” for the user and not restriction, the user being allowed toenter a new value.

3.6.3 Case Similarity

The similarity calculation algorithm implemented by The Easy Reasoner does notconsider vocabulary; this raises terminology problems, which we have tried to avoidby restricting the terminology used in some fields (see section 3.6.2), and offering aconsiderable level of detail in the case structure as a whole. However, terminologyproblems might still occur, due to the difficulty to further structure some of thefields, like Problem Description, Constraints, considering the scope of decisionsupport required by the end user and the general nature of the application. In thesefields, the risk is that the same information described with different words, (i.e.,different people might describe the same problem in the different ways), isdisregarded by the retrieval algorithm. This affects the results in the sense that therelevance of the case might not be the same as perceived by the user, i.e. the casecan be retrieved with a lower similarity percentage. To increase the probability ofretrieving the most similar case in the results of a query, our system uses the moretextually restrictive fields, within the case structure, containing information aboutthe context of the problem (e.g. Program main phase, Client operating environment,etc). However, due to the technical complexity of the algorithm used in The EasyReasoner implementation, the similarity percentage offered for each case in theresults might not reflect the ‘real’ similarity of the cases, as understood by the user.The software does not show enough transparency and can affect user’s trust.

3.6.4 Case Adaptation

The case evaluation and adaptation functions were not developed in our application.There were a number of reasons for this, as discussed below.

In order to perform an evaluation and adaptation of the case(s) retrieved, either arule-based production system should be used, or the need for adaptation should bedecreased. In CE, decision-making problems are ill defined and ill structured, whichmakes it difficult to formalise the knowledge and build the rule-based model. A rule-based model has to be context specific, hence this would make the product lessgeneric and thus considerably increase the cost of customisability. Currently,

customisability is solved through case representation, designed to fit or adapt to anycompany context. Case adaptation would require implementing a separate rule-basedmodel for every company context, with additional support from knowledgeengineers, which means very high development and maintenance cost for the CBRsystem.

Decreasing the need for adaptation by improving the retrieval capabilities wouldrequire the developers (us) to build their own retrieval mechanism, designed fortheir problem. But since they are using a commercial CBR tool, which has its ownretrieval mechanism, this approach for adaptation was not appropriate.

So, without the automated adaptation function our system requires additional humanreasoning, i.e. increased participation of the user in evaluating the solution anddeciding if it can be reused. The problem of reduced system performance orfunctionality can be overcome through extensive case collection (the more cases inthe database, the better results) and via more effort in refinement of the system.

Like any decision support tool, a CBR system is limited to ‘suggest’ a solution andnot to assert that ‘this is the solution to the problem’. Hence, even an adaptedsolution would have to be filtered by the human reasoning before being applied. Aspractice has proved, in some cases the results of an adapted solution would not bejustified to a sceptical user. These results of adaptation must be, therefore, validated,like in any other decision support system. The best validation is through a practicalimplementation of the solution provided by the adapted case. The risk of failure ofthis implementation is not decreased by adaptation [11] and the cost of failure doesnot justify the practical validation, especially in costly commercial projects.

Thus, with adaptation, the degree of user involvement in the actual reasoningdecreases. The human adaptation implies more participation and greater effort fromthe user, whereas the automatic adaptation improves the relevance of the results andminimises the user effort. The final decision belongs, however, to the user.

Recognising that practical retrieval technologies are available, but the generaladaptation problem remains extremely difficult for CBR systems, experts in bothCBR research [12] and applications [11, 13, 14] agree that the best use of CBR is asadvisory systems that rely on the user to perform evaluation and adaptation. ManyCBR systems simply dispense with adaptation, replacing the retrieve-evaluate-adapt-learn cycle with retrieve and propose cycle [12], i.e. the best case retrieved isproposed to the user, who will evaluate and adapt the solution.

4 Application Benefits

The reason for applying a CBR decision support system was to improve the CEprocess or practices by improving the decision-making process. The success of CElies in the achievement of collaboration between different team members. We havedeveloped a decision support tool that provides factual information in a structuredand context relevant format, and which encourages human intervention and

discourse. This would enable the team to move towards a more CE conduciveculture. This is achieved by empowering the project managers and team members byremoving the reliance on the so called hard to get experts and providing the hard toget corporate knowledge hidden in company archives in an immediate, structured,coherent and reader friendly manner.

The other benefit is in improved design capability in terms of enabling engineers toevaluate different design options under different constraints and interrelated criteria,i.e., Design for X (DFx) type of studies. The highly structured nature of the casescombined with the what-if analysis enables engineers and managers to investigatedifferent scenarios. Whilst considerable efforts can be put into modelling andcapturing the expertise to carry out the automated DFx evaluation of design, theseevaluations can be computationally expensive. In this respect they would fail toprovide the benefit of timely feedback to the designer which could be realisedthrough the use of cases [15]. Using CBR could result in considerable cost and timesavings in the NPD process.

However the success of CBR, which can itself be costly to develop and implementdepending on the scale of application and tool used, requires considerable training ofthe managers and team members towards documenting and sharing their experiencesand hence knowledge in a manner which others can use. Additionally there needs toa commitment to maintain and improve the knowledge base. An ‘informationsharing and knowledge friendly’ culture hence needs to be instilled for such decisionsupport tools to succeed.

5 References

[1] J. Riedel and K. S. Pawar. The Strategic Choice of Simultaneous Versus SequentialEngineering for the Introduction of New Products. International Journal of TechnologyManagement, Special Issue on Manufacturing Strategy, 1991.

[2] L. Trygg. Simultaneous Engineering: A Movement or an Activity of the Few?International Product Development Management Conference on New Approaches toDevelopment and Engineering, Brussels, 18-19 May, 1992.

[3] S. G. Shina. Successful Implementation of Concurrent Engineering Products andProcesses. Van Nostrand Reinhold, New York, 1994.

[4] R. W. Hanssen. Reducing Delivery Times in Engineer-To-Order Firms by Using theConcepts of Concurrent Engineering. Proceedings of the 4th International Conference onConcurrent Enterprising (ICE’97), The University of Nottingham, October 8-10, pp.495- 508, 1997.

[5] V. Walsh et al. Winning by Design. Blackwell Publishers, 1992.[6] F. Weber et al. User Requirements Definition. CODESCO Deliverable D11, ESPRIT

Project No. 25455, 1998.[7] CODESCO Project Programme. ESPRIT Project No. 25455, 22 Sept., 1997.[8] D. B. Leake. CBR in Context: The Present and Future. In D. Leake’s Case-Based

Reasoning– Experiences, Lessons, & Future Directions. AAAI Press / MIT Press, 1996.[9] S. Benett. Guide to Management and Technology. Web:

http://www10.geocities/SiliconValley/Pines/1814essay501.htm, 1996.[10] J. Kolodner. Case-Based Reasoning. Morgan Kaufmann Publishers, San Mateo, CA,

1993.

[11] W. Mark, E. Simoudis, and D. Hinkle. Case-Based Reasoning: Expectations andResults. In D. Leake’s Case-Based Reasoning – Experiences, Lessons, & FutureDirections. AAAI Press / MIT Press. 1996.

[12] J. L. Kolodner. Improving Human Decision-Making through Case-Based DecisionAiding. AI Magazine 12(2), pp. 52-68, 1991.

[13] R. Barletta. A Hybrid Indexing and Retrieval strategy for Advisory CBR Systems Builtwith REMIND. In proceedings of the Second European Workshop on Case-BasedReasoning (EWCBR), pp. 49-58, Chantilly, France, 1994.

[14] H. Kitano and H. Shimazu. The Experience Sharing Architecture: A Case study incorporate-Wide Case-Based Software Quality Control. In D. Leake’s “Case-BasedReasoning – Experiences, Lessons, & Future Directions. AAAI Press / MIT Press,1996.

[15] Macdonald R. Transforming heuristics into cases: an evolutionary approach to theconstruction of multi-criteria decision support systems. Published on the web page ofthe Multi-media Communications Group, Department of Psychology, University ofGlasgow, Glasgow, Scotland, 1998. (http://www.mcg.gla.ac.uk/staff/rory/wec2.html)