Integrating Decision Support Systems: Expert, Group, and

Collective IntelligenceSteve Diasio* & Núria Agell

ESADE Business School- BarcelonaGREC Research Group

*This research has been partially supported by the AURA research project (TIN2005-08873-C02), funded by the Spanish Ministry of Science and Information Technology and the Commission for Universities and Research of the Ministry of Innovation, Universities, and Enterprises of the Government of Catalonia.

IC’ AI 09 Las Vegas, 2009

Road Map

• Introduction and Motivation• Framework for Integration• Terms and Concepts• Leveraging Expertise in

Decision Support Technology– Expert Systems (ESs)– Group Decision Support

Systems (GDSSs)– Collective Intelligence

Tools (CI Tools)• Enhancing Decision-Making

and CI Tools• Conclusions and Future

Work

Empty trading pit @ CBOT

Introduction and Motivation

• Organizations today face a changing environment; – external conditions change rapidly (Ilinitch et al, 1996).– organizational structures flat and dispersed (Malone, 2006).– traditional roles of experts have been “squeezed” or of decreased

importance (Mauboussin, 2008).• Today’s new environment places a premium on collaboration

creating renewed interest in decision support technology to survive and remain competitive (Hamel & Breen, 2008).

• Information technology is playing an increasing role in facilitating a firm’s success and is woven thread in the fabric of the organization (Zammuto et al, 2007).

• The paper aims to understand how integration of expert systems (ESs), group decision support systems (GDSSs), and collective intelligence tools (CI tools) can enhance decision-making.

Framework for Integration

• Abundance of decision support tools at their disposal.• Tools have been independently built (Turban & Watkins,

1986) for individual problems but be flexible to adapt to the changing conditions and needs.

• Individually shown advantages of using such systems, however have not extended or offered in theory or practice an integrated system that supports organizational needs in expertise for decision-making.

Expert Systems GDSSs CI Tools

Integrated System

Proposed integrated support system

• Abundance of decision support tools at their disposal.• Tools have been independently built (Turban & Watkins, 1986) for

individual problems but be flexible to adapt to the changing conditions and needs.

Expert System GDSSs CI Tools

Integrated System

Framework for Integration

Proposed integrated support system

• Individually shown advantages of using such systems, however have not extended or offered in theory or practice an integrated system that supports organizational needs in expertise for decision-making.

Terms and Concepts

What is Expertise?• Multi-dimensional (Sternberg,

1997) with expert knowledge as the essential part (Tynjala, 1999)

• Short supply and difficult to represent

• Highly specialized or domain specific (Chi, Glaser, & Farr, 1988)

• Skills honed through practice (Jackson, 1999)

• Perform consistently more accurate in relation to others (Hartely, 1985)

Practical Knowledge Self-r

egula

tive

Know

ledgeFo

rmal

Kno

wled

ge

Expert Knowledge Dimensions

Expertise in Law

Formal Knowledge Practical Knowledge

Self-regulative Knowledge

•Reflective skill•Evaluation of action•Monitor argument and presentation to jury

•Factual knowledge•Learning of explicit information•In school or cases

•Intuition•Experience in legal setting•Tacit and difficult to express

Lawyer Expertise

Expertise by Means of Technology

• Expertise not limited to humans

• Technology built to capture knowledge or represent expertise (Barton, 1987; Liou & Nunamaker, 1990; Smith, 1994)

• Level of expertise can be augmented by increasing the amount of participants in the decision-making process

Expertise in Design

Level of Expertise in Systems Design

Number of People

Expert Systems

GDSSs

Collective Intelligence Tools

Objective: To represent expertise to its users for decision-making when a human expert can not be found or is in short supply.

Playing a critical role for organizations and are a source for competitive advantage (Gill, 1995).

Contributing to decision-making through their representation of knowledge and reasoning of human experts (Weiss & Kulikowski, 1984).

By mimicking and replicating the cognitive process of a human expert, novice users can be supported to perform as well as experts (Cascante et al, 2002).

ES are a technology that facilitates learning through the transfer of tacit and explicit knowledge (Yoon et al., 1995; Gregor & Benasat, 1999).

Leveraging Expertise Expert Systems

Attributes:

Objective: To capture the knowledge and contribution from the individual users to facilitate solutions to problems.

Occupies the center point for the aggregation of information and expertise from each participant.

Support the changing organizational structure, project basedteams, dispersed workforce, and greater emphasis on collaboration.

Aided groups to deal with to the changing dynamics characterized by greater knowledge, complexity, and turbulence (Huber, 1982; 1984).

Shown to reduce time, costs (Gallup, 1985), foster collaboration, communication, deliberation, and negotiations (Kull, 1982).

Leveraging Expertise Group Decision Support Systems

Attributes:

What is Collective Intelligence?

• The collective judgment of group can predict or forecast better than experts or groups of experts (Surowiecki, 2004)

• Diverging from traditional thought- high levels of expertise are the best source for decision-making

• Including many people in decision-making by harnessing lower levels of expertise for peak solutions (Page, 2007)

Objective: To facilitate the summative body of knowledge,information, and resources of its users.

Democratize decision-making by including many people in and outside the organization into the information gathering and decision-making process.

Prediction markets, incubates information scattered around the organization or network that allows non-experts to produce expert like results.

Challenges traditional roles of experts, may change answer givers to inquiry mediators in effort to harness the knowledge of the masses in decision-making.

Offer an additional tool in decision-making.

Leveraging Expertise Collective Intelligence Tools

Attributes:

Enhancing Decision-Making and CI Tools

• Past attempts have made steps (Aiken et al. 1991; Turban & Watkins, 1986).

• Opportunities for system integration to solve a wider spectrum of problems.

• AI techniques to CI Tools– Transforming from

passive to active agents– Intelligent components

to increase participation – Managing interaction

and collaboration between users

Ill- Structured

Problem Structure

Many

ES

Group Size SupportedFew

Well- Structured

GDSS

CI Tools

DSS

Decision Support Technologies *

*Figure adapted from Aiken et al. 1991

Differences Between ES, GDSSs, CI Tools

Attributes ES GDSS CI TOOLS

Objective Replicate or mimic human expertsFacilitate solutions for a group of people

To sum the knowledge and information of many people

Who makes the recommendation (decision)?

The system or heavily weighted if human is involved

The group and/ or systems through ranking The System/ Tool

Major orientation (characteristic)Transfer of expertise (human-machine-human) Build group consensus

Transfer of hard to find information or qualitative to quantitative data

Nature of support Individual or group Group Individual or group

Problem area characteristic Narrow domain Semi/ Unstructured, broad Limited variability

Type of problem treated Repetitive Unique/ not often / importantForecasting/ dispersed collaborators/ Probabilistic

Reasoning capability Yes (deduction) NoYes (depending on the tool (induction)

Assumptions Closed-world Limited to users boundaries Changing

Expertise Level or In-depth knowledge of problem Specific/ Expert Level Dependent on task or problem

All levels including learning capacity with use

Figure 4 Differences between ES, GDSS, CI ToolsAdapted from Aiken et al, 1991]

Shown Indicated

ExploredHighlighted

Conclusions

an evolutionary perspective of expertise supported by decision support technologies.

how organizational use of expertise is changing which reflects the new roles of experts and non-experts in decision-making

how organizational expertise in short supply can be augmented

issues of design for integration with existing decision support technology

Thank You!

Steve Diasio & Núria Agell{stephen.diasio; nuria.agell} @esade.edu

ESADE Business School- BarcelonaGREC Research Group

References[1] M. Aiken, O. Liu Sheng and D. Vogel, “Integrating expert systems with group decision support systems”,

ACM Transactions on Information Systems (TOIS), Jan. 1991. (9)1, 75-95.[2] L. Bañón and K. Schmidt, “CSCW: Four characters in search of a context”, Proceedings of the First

European Conference on Computer-Supported Cooperative Work - EC-CSCW Gatwick, London, Sept., 1989, 358-372.

[3] A. Barton, “Experiences in Expert Systems”, The Journal of the Operational Research Society, The Practice of OR in Central Government Oct., 38(10), 1987, 965-974.

[4] J. Berg and T. Rietz, “Prediction Markets as Decision Support Systems”, Information Systems Frontiers, 5(1), 2003, 79-93.

[5] J. Berg, F. Nelson and T Rietz, “Accuracy and forecast standard error of prediction markets”, Working Paper, The University of Iowa, 2001.

[6] E. Bonabeau, “Decisions 2.0: The Power of Collective Intelligence”, MIT Sloan Management Review. Winter 50(2), 2009.

[7] B.G. Buchan, D. Barstow, R. Bechtel, J. Bennett, W. Clancey, C. Kulikowski, T. Mitchell and D.A. Waterman, “Constructing and Expert System”, Building Expert Systems, Hayes-Roth, F., Waterman, D.A. and Lenat, D.B. (eds), 1983. pp 127-167, Addison-Wesley, Reading, MA.

[8] L. Cascante, M. Plaisent, P. Bernard and L. Maguiraga, “The impact of expert decision support systems on the performance of new employee”, Information Resource Management Journal; Oct-Dec 15(4), 2002, 67-78.

[9] M. T. H. Chi, R. Glaser, and M. J. Farr, eds., The Nature of Expertise, 1988. Hillsdale, NJ: Erlbaum.[10] B. Cowgil, J. Wolfers, and E. Zitzewitz, “Using Prediction Markets to Track Information Flows: Evidence

from Google”, 2008, Working Paper.[11] K. Crowston and T. W. Malone, “Intelligent software agents”, Byte Magazine, Dec., 1988, 267-272.[12] G. DeSantics and R. Gallup, “A Foundation for the Study of Group Decision Support Systems”.

Management Science, 33(5), 1987, 589-609.[13] S. Diasio, “Collaborative Fusion: Expert, Group, and Collective Intelligence in Decision-making

Support”, unpublished paper, 2008.[14] R. Dye, “The Promise of Prediction Markets: A Roundtable”. McKinsey Quarterly (2) 2008.

References[15] E.A Feigenbaum, “The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering’,

International Joint Conference on Artificial Intelligence, 1977, 1014-1029.[16] B. Gallup, “The impact of task difficulty on the use of a group decision support system”. Ph.D. dissertation,

Dept. of Information and Decision Sciences, Univ. of Minnesota, 1985.[17] M. J. Gannon, Organizational Behavior: A Managerial and Organizational Perspective, Boston: Little, Brown,

1979.[18] T. Gill, “Early Expert Systems: Where are they now?”, MIS Quarterly. March, 1995.[19] R. Hanson, “Decision Markets”, IEEE Intelligent Systems, 14, 1999, 16-19.[20] R. Hartley, “Expert System Methodology: A Conceptual Analysis”, International Journal of Systems Research

and Information Science, 1, 1985, 11-22.[21] T. Ho, and K. Chen, (2007). “New Product Blockbusters: The Magic and Science of Prediction Markets”,

California Management Review. Fall, 50(1), Berkeley, California, 2007.[22] J. Howe, Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, New York, Crown

Business, 2008.[23] G. P. Huber, “Group decision support systems as aids in the use of structured group management

techniques”, In DDS-82 Conference Proceedings, 1982, 96-108.[24] G. P. Huber, “Issues in the Design of Group Decision Support Systems”, MIS Quarterly, Sept. 8(3), 1984,

195- 204.[25] P. Jackson, Introduction to Expert Systems, (3rd) ed, Pearson Addison Wesley, 1999.[26] D. Kull, “Group decisions: Can a computer help?”, Computer Decisions, vol. 15(5), 1982, 64-70.[27] Y. Liou, and J. Nunamaker, “Using a Group Decision Support System Environment for Knowledge Acquisition:

A Field Study”, Proceedings of the ACM SIGBDP conference on Trends and directions in expert systems. Orlando, Florida, United States, 1990, 212 – 236.

[28] D. S. Murphy, and S. A. Yetmar, “Auditor evidence evaluation: Expert systems as credible sources”, Behavior and Information Technology, 15, 1996, 14-23.

[29] S. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Princeton University Press, 2007.

[30] E. H. Shortliffe, and B. Buchanan, “Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project”, 1984.

References[31] E. H. Shortliffe, “Computer-based medical consultation MYCIN”, American Elsevier, 1976.[32] H. Simon, Administrative Behavior: A Study of Decision- Making Processes in Administrative

Organizations, 4th ed. The Free Press, 1997.[33] B. Smith, Collective Intelligence in Computer-Based Collaboration, Hillsdale, NJ: Lawrence Erlbaum,

1994.[34] R. J. Sternberg, “Cognitive conceptions of expertise”, In P.J. Feltovich. K. M. Ford and R. R. Hoffman.

Expertise in context. Human and Machine, Menlo Park, CA: AAAI Press/ The MIT Press, 1997. 149-162.[35] J. Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective

Wisdom Shapes Business, Economies, Societies and Nations, Little, Brown, 2004.[36] P. Tynjala, “Towards expert knowledge? A comparison between a constructivist and a traditional

learning environment in the university”, International Journal of Education Research. 31, 1999, 357-442.

[37] J.P. Shim, M. Warkentin, J.F. Courteny, D.J. Power, R. Sharda, and C. Carlsson, “Past, Present, and Future of Decision Support Technology”, Decision Support Systems, 33, 2002, 111-126.

[38] S. Weiss and C. A. Kulikowski, A Practical Guide to Designing Expert Systems, Chapman and Hall Ltd, 1984.

[39] S. E. White, J. E. Dittrich and J. R. Lang, “The Effects of Group Decision Making Process and Problem Situation Complexity on Implementation Attempts”, Administration Science Quarterly, 25(2), 1980, 428-440.

[40] J. Wolfers and E. Zitzewitz, “Prediction Markets in Theory and Practice”, In S. Durlauf and L. Blume (eds) The Palgrave Dictionary of Economics, McMillan, 2nd ed, 2006

[41] M. Woodridge and N.R. Jennings, “Intelligent Agents: Theory and Practice”, The Knowledge Engineering Review, 10(2), 1995, 115-152.