Organizational schemata of e-portfolios for fosteringhigher-order thinking

Shouhong Wang & Hai Wang

Published online: 4 August 2010# Springer Science+Business Media, LLC 2010

Abstract The information technology of online e-portfoliosystems have been widely used during the past several yearsalong with the diffusion of electronic teaching-learningsystems. However, for the time being e-portfolio is viewedmore as an assessment tool or a showcase tool, but less as anactive learning tool. The current generic e-portfolio systemsstore artifacts in the chronological order on the course basis,providing few facets for active thinking. The question of howwe can make e-portfolio a useful learning tool to improvestudents’ learning outcomes is still open to research. Amongvarious students’ learning outcomes, higher-order thinkinghas become an important outcome of education. One vision ofeducation evolution is to change the modes of thinking ofstudents. This study is to meet the challenge of e-portfolios byinvestigating a significant research question: how e-portfolioscan be used as a learning tool for students to foster higher-order thinking. Specifically, this study proposes an ontologicalapproach to organizational schema of e-portfolios so thate-portfolios can be logically and dynamically organized intothinking-driven networks. The ontological schemata can serveas visible maps for the virtual e-portfolios repository shared byall teachers and students to foster higher-order thinking. Acase study that implements a prototype of organizationalschemata of e-portfolios demonstrates the usefulness of theproposed approach for fostering higher-order thinking.

Keywords Semantic Web . System analysis and design .

User-computer interface . Object-oriented . E-portfolio .

Higher-order thinking . Ontology . E-learning .

Learning objects . Business education .

Information systems education

1 Introduction

Information technology has significant impacts on the envi-ronment of education and approaches to teaching and learning(Zhang and Nunamaker 2003; Barolli et al. 2006; Carchioloet al. 2007; Shen et al. 2008). Recently, more and morestudents and faculty members are using e-portfolio systems.There will be massive pieces of educational resources storedin those systems. E-portfolios are supposed to serve threepurposes: assessment, showcase, and learning (Greenberg2004). For assessment purposes, e-portfolios include rubrics-based documentations and feedback from teachers. Forshowcase purposes, e-portfolios present artifacts of accom-plishments and lifelong career development. For learningpurposes, e-portfolios can be useful for on-going reflection.

The current commercial or open source e-portfoliosystems have been successfully used for assessment andshowcase, but have not been effectively applied toenhancing students’ learning (Zhang et al. 2007). This ismainly because generic e-portfolio systems are more or lesslearning subject independent. On the other hand, usefullearning portfolios must be learning-subject specific. Tomake generic e-portfolio systems more useful for enhancingstudents’ learning, an interactive layer between the users(teachers and students) and e-portfolio repository must bedeveloped to facilitate students’ learning. This middlewarelayer supports dynamic organizations of e-portfolios foractive learning purposes.

S. Wang (*)University of Massachusetts Dartmouth,North Dartmouth, MA, USAe-mail: swang@umassd.edu

H. WangSaint Mary’s University,Halifax, NS, Canadae-mail: hwang@smu.ca

Inf Syst Front (2012) 14:395–407DOI 10.1007/s10796-010-9262-0

Higher-order thinking has been an important learningoutcome in education. For instance, in 2003 AACSB(Association to Advance Collegiate Schools of Business),the global accreditation agency for business education,adopted a new set of standards for business schoolsaccreditation. The standards of Management of Curriculumsuggest that reflective thinking skills should be animportant outcome of business education (AACSB 2003).The challenge of reflective thinking raises a significantresearch question: how e-portfolios can be used as alearning tool for students to foster higher-order thinking.This study is to apply semantic Web techniques to addressthis challenge. Specifically, this study proposes user-drivenontology-based organizational schemata of e-portfolios thatcan support students’ reflection and higher-order thinking.The ultimate objective of this study is to make contributionto information systems design science (Walls et al. 1992;Hevner et al. 2004; Gregor 2006; Jones and Gregor 2007)through the demonstration of the feasibility of using thedesign principles to develop an information system artifactfor e-portfolios systems.

This paper is organized as follows. First, e-portfolio and itschallenge are brief overviewed. This enables us to formulatethe objectives of this study. Second, higher-order thinking andthe support of e-portfolios are discussed. Third, ontology isdiscussed as a tool for making thinking visible and teachable.Fourth, a model of user-driven ontology-based organizationalschemata of e-portfolios is proposed. This model provides thestructures of interactive systems between e-portfolios andusers to facilitate students’ higher-order thinking. Fifth, a casestudy of a prototype of interactive system for e-portfoliosbased on the proposed ontological model is described. Finally,our conclusion is presented.

2 E-portfolio and its challenges

2.1 E-portfolio and e-portfolio systems

A portfolio is a systematic and purposeful collection of workand achievement documentations (Drier 1997). A learningportfolio is an evidence-based tool that documents studentwork and engages students in a process of continuousreflection and collaborative analysis of learning (Zubizarreta2004). E-portfolios are highly personalized, customizable,and Web-based files which document learning portfolios anddemonstrate individual and collaborative learning processes(McCowan et al. 2005). An e-portfolio system is a Web-based repository management system that stores students’learning documents (so-called artifacts) such as academicrecords, essays, project reports, assignments, assessments,and personal and professional development related contents.Students use e-portfolio systems to present artifacts, receive

feedback from instructors and advisors, and communicatewith each other.

There are many commercial, non-profit organizational,and open-source e-portfolio systems, such as Open-SourcePortfolio (OSP) (OSP 2005), Chalk & Wire (CW 2010),KEEP toolkit (KEEP 2010), foliotek (2010) and Task-Stream (2010). While there are high variations of userinterface design among these e-portfolio systems, thefunctionalities of current competitive e-portfolio systemsare about the same and include artifacts editing anduploading, commenting and assessing on student work,communicating and sharing within groups, showcasegenerating, and administrative reporting.

2.2 Benefits and challenges of e-portfolios

The e-portfolio systems have brought great benefits to theeducation community. E-portfolios are stored online and havegreat accessibility for the portfolio owners themselves, teach-ers, colleagues, and employers (Bruder 1993; Bushweller1995; McCowan et al. 2005). E-portfolios are a mechanismfor students and education institutions to improve anddemonstrate their teaching/learning skills and to dis-play competencies to the society (Lumsden et al.2001). E-portfolio systems enable administrations at alllevels to survey and to conduct comprehensive assess-ment of teaching and learning accomplishments (Barrett1994).

An e-portfolio artifact is a unit of digital resource thatcan be used to support learning, and thus is a learningobject (Wiley and Edwards 2002). Along with the increas-ing use of e-learning systems, learning objects becomeincreasingly valuable and, at the same time, the manage-ment of learning objects repository becomes complicated(Collis and Strijker 2003; Cohen and Nycz 2006; Singh etal. 2007). There have been metadata standards for learningobjects, such as those proposed by IMS Guide (IMS 2006),Dublin Core (DC 2010), and IEEE LTSC (IEEE LTSC2010). These standards are used to represent individuallearning objects at the collection level which is similar tolibrary catalogue systems. However, to use learning objectsto support teaching and learning for a specific field,knowledge schema must be applied to the learning objectsrepository for the domain (Koohang 2004; Harman andKoohang 2005). This is because learning objects can beorganized in a variety of ways depending upon complexintra-context and inter-context (Wiley 2000). When alearning objects repository is huge and is distributed onthe Internet, the use of meta-data and keywords only tosearch the needed learning objects is inefficient andineffective since much potential associations with variouslearning aspects are bypassed (Mustaro and Silveira 2006).This has lead to approaches to semantic Web applications

396 Inf Syst Front (2012) 14:395–407

that model the relationships between learning objects usingformal ontologies (Sicilia and Lytras 2005).

Ontology techniques have been applied to repositories oflearning objects (Snae and Brueckner 2007; Zouaq et al.2007; Wang 2008). Ontologies implement dynamic organ-izations of learning objects based on application conceptsand diversified needs of teaching and learning. Accord-ingly, there is a great need for ontological models thatshow the relationships between e-portfolio artifacts andstudents’ higher-order thinking modes. These models canbe used by teachers and students to store, to search, toaccess, and to update e-portfolios effectively towardsreflection and higher-order thinking. It is the centraltheme of this study that higher-order thinking orientedorganizational schemata of e-portfolios are needed toutilize e-portfolios as an active thinking tool. Currently,the organizations of artifacts in e-portfolio systems aremore or less static, staying on the individual course basis.On the other hand, the access of e-portfolio artifacts forreflection and active learningmust be spontaneous in responseto the modes of thinking. However, few research reports havediscussed ontologies for reflection and active thinkingthrough the use of e-portfolios. The motivation of thisstudy is to investigate how ontology techniques can beused for the implementation of dynamic organizationalschemata of e-portfolios to foster students’ higher-orderthinking.

3 Higher-order thinking and its paradigms

3.1 Higher-order thinking

Higher-order thinking is more than simple memorizationand comprehension, and involves a variety of cognitiveprocesses, such as generating ideas, exploring consequen-ces, reviewing options, monitoring progress, and so on(Perkins et al. 1993a). In this study, higher-order thinking,reflective thinking, integrative thinking, and active thinkingare interchangeable terms.

The taxonomy of higher-order thinking has not been madeclear. Skeptically, as higher-order thinking is so complicatedin general, any taxonomy is unlikely to be applicable to allsubjects. In some cases, discipline-specific and skill-specificknowledge plays an important role in higher-order thinking. Incontrast, for career development or self-regulation, higher-order thinking is generally non-discipline-specific and maynot involve any discipline-specific knowledge.

3.2 Non-discipline-specific higher-order thinking

Along with the proliferation of e-portfolio systems, therehave been discussions on non-discipline-specific higher-

order thinking through the use of e-portfolios. Essentially,three major paradigms of non-discipline-specific and non-skills-specific higher-order thinking are discussed in theliterature: career development, academic accomplishment,and extra-curricular learning (Annis and Jones 1995;Batterbee and Dunham 2004; Zubizarreta 2004). Non-discipline-specific and skills-specific higher-order think-ing paradigms include: self-regulation and motivation(Kirkwood 2000).

Although higher-order thinking emphasizes generalthinking strategies and abilities across diverse situations,discipline-specific knowledge can guide higher-order think-ing that is relevant to the particular discipline (Ericsson andSmith 1991). Here, we place the focal point on the businesseducation field while addressing the central theme of thisstudy.

3.3 Higher-order thinking in business education

Critical thinking, design thinking, and systems thinking arethree general management thinking paradigms that arerelated to problem solving skills in the business education.

3.3.1 Critical thinking

Critical thinking has been widely discussed in the businesseducation literature (e.g., Kurfiss 1988; Jenkins 1998; Pageand Mukherjee 2007; Peach et al. 2007), but is oftenoverstated to be overall higher-order thinking. In this study,critical thinking is the ability to explore a problem, integrateall the available information about it, derive a decision, andjustify the decision (Warnick and Inch 1994; Plous 1993;Kida 2006). Simon (1976) has laid the foundation for thetheory of behavioral and cognitive processes engaged indecision making. Critical thinking involves gatheringappropriate information, evaluating alternative answerspertinent to this information, and choosing the answer thatis best supported by the information.

In terms of cognitive style, critical thinkers tend to bethinking (as opposed to feeling), extroversion (as opposedto introversion), judgment (as opposed to perception), andsensing (as opposed to intuition) oriented (Myers 1962) inorder to make logical, analytical, objective, and empiricaldecisions.

3.3.2 Design thinking

Under a design-thinking paradigm, people would beencouraged to think broadly about problems, develop adeep understanding of issues, and plan a process toimplement a good idea. Design thinking is different fromcritical thinking in that design thinking is process-orientedwhile critical thinking is judgment-oriented. For instance,

Inf Syst Front (2012) 14:395–407 397

optimization approaches emphasize more on critical think-ing, but less on design thinking. Design thinking resultsfrom the nature of design work: a project-based work flowaround problems (Dunne and Martin 2006).

The literature (e.g., Peirce 1997; Simon 1996; Bolandand Collopy 2004) has addressed numerous aspects ofdesign thinking. In terms of cognitive aspects, designthinking includes induction, abduction, and deductionmental processes. Generally, to facilitate design thinking,project-oriented process models are needed (Dunne andMartin 2006). The major task of initiating design thinkingthen becomes the identification of processes to implement aplan.

3.3.3 Systems thinking

Systems thinking paradigm originated from organizationallearning theories (Senge 1990; Argyris and Schon 1988).The essence of systems thinking is focused attention todynamic interaction factors of organizational networks(Thurston 2000). The system thinking approach allowsmanagers to visualize a managerial problem as a system ofcomponents (structures, entities, events, and factors), and tounderstand how interdependent components impact eachother.

Although there have been research into the cognitiveperspective of systems thinking (e.g., Kim 1993; Eysenck1993), few mental models for systems thinking have beendeveloped beyond the traditional psychological theories ofshort-term memory, long-term memory, and learning andrecalling. Nevertheless, a pragmatic approach to fosteringsystems thinking is the use of system language (Eliot1987; Senge 1996). A system language represents andcommunicates the complicated interplay among the sys-tem components.

3.4 A summary of higher-order thinking and potentialsupport of e-portfolios

We have discussed the higher-order thinking paradigms.Each of these thinking paradigms involves multiple levelsof thinking in the Bloom’s educational goal framework:knowledge, comprehension, application, analysis, synthe-sis, and evaluation (Bloom 1956). Clearly, the cut-linesbetween the higher-order thinking paradigms can never besharp. Also, it is not the intention of this study to identifyall types of higher-order thinking paradigms. The focalpoint of this discussion is to gain more understanding aboutthe different modes of higher-order thinking and toinvestigate how we can use e-portfolios to support teachinghigher-order. Generally, the relationships between thediversified higher-order thinking paradigms and the supportof e-portfolios can be described in Table 1. As illustrated in

Table 1, e-portfolios can support higher-order thinking inmany ways. This study concentrates on the organizationalschemata of e-portfolios for teaching and learning higher-order thinking.

4 Ontologies of e-portfolios: Making thinking visibleand teachable

4.1 Ontology in the context of learning objects

Ontology is a science that studies explicit formal specifi-cations of the terms in the domain and relations amongthem (Gruber 1993). In the general philosophical term, anontology is a specification of a conceptualization (Gruber1995). In the semantic Web field, an ontology is typically anetwork of semantically related objects for a specificdomain. An ontology allows people to share commonunderstanding of the subject domain.

According to Resource Description Framework (RDF)(W3C 2010), a primitive ontology is a triple containing asubject, an object, and a predicate (relationship) (seeFig. 1a). Its special form that represents the reciprocalrelationship between two learning objects (dual subject andobject) is shown in Fig. 1b. A large ontology for an entiredomain is a composition of a set of primitive ontologies.Given the complexity of learning objects structures ingeneral, a learning object itself can be represented by anontology. An ontology for a learning objects repository is aconceptual network of all related learning objects thatshows the semantic relationships between the learningobjects in the application domain.

4.2 Ontology presents the object-oriented visionof e-portfolios

All learning objects are natural objects. Rationally,e-portfolios can be represented by an object-oriented modelthat represents the relationships between the e-portfolioobjects (artifacts). The premise of object-oriented mod-eling is that objects are grouped into categories or classesfor the application domain (Wang 1999a). Classes areorganized into hierarchies in which the sub-classes inheritproperties from their super-class. For instance, the sub-classesof e-portfolio objects inherit meta-data from their super-class.A sub-class can inherit from multiple super-classes.Inheritance relationships result in static connectionsbetween e-portfolio objects. In addition to inheritancerelationships, the object-oriented paradigm applies so calledmessage sending from one class to another to make dynamicconnections between the classes. These messages accentuatethe dynamic relationships between the classes that representcontingent access paths to e-portfolios. All static and dynamic

398 Inf Syst Front (2012) 14:395–407

relationships between the classes specify the semanticproperties of the entire sets of e-portfolio objects.

The ontological approach is a powerful modeling ap-proach; however, without a domain analysis for particulartypes of applications, the ontological approach remains avirtual philosophy, rather than a concrete technique for objectssharing (Devedzic 2004). To build ontologies based on themethodology progression, ontologies of e-portfolios mustpresent the object-oriented vision. The task of a domainanalysis for the construction of an ontology is to actualizeclasses of e-portfolio objects and their semantic relationships,as illustrated later in the case study of this paper.

4.3 Ontological representation of organizational schematafor the e-portfolios

In the semantic Web area, ontologies serve as models of theinterfaces between the user and the learning objectrepository to provide views of learning objects in variousperspectives to enhance the learning objects repositoryusability for diverse application domains (Smrz 2004;Namuth et al. 2005). The ontology of organizationalschemata is envisaged as knowledge structures that fit theindividual applications. For instance, an e-portfolio systemcan have three different views: students’ view, instructors’view, and administrators’ view. The three views of theorganizational schemata can be represented by a commonontology that describes procedures of creating and editingartifacts, interaction processes of assessing and comment-ing, and functions of reporting. The structures of all menus

and linkages constitute the ontology which might not beexpressed explicitly.

Ontologies have been with us for a quite long time. Forinstance, ER (entity-relationship) chart is a general type ofontology for relational databases. In comparison with ERcharts for relational databases, ontologies for e-portfolios arecomplicated due to the complex properties of e-portfolios andthe richness of semantics in the context of assessment andreflection.

4.4 Ontological schemata make thinking visibleand teachable

Although the real thinking in the human brain remains by andlarge an enigma, researchers have been seeking instrumentsfor fostering higher-order thinking. Research (Quillian 1968;Perkins et al. 1993a, b) has made connections betweenontology and thinking that ontological expressions ofdomain specific knowledge and thinking processes can makesignificant contributions to teaching higher-order thinking.The ability of organizational representation makes ontology atool of constructing schemata for e-portfolios. Through theuse of ontological schemata, one can make thinking morevisible than it usually is in classrooms by providing studentstheir own e-portfolios to build on and learn from. Ontologicalschemata surface many opportunities for thinking duringlearning subject domains. They enable us to describe thinkingroutines (Ritchhart 2002) for higher-order thinking process,and to make higher-order thinking visible as well asteachable.

4.5 An illustrative example of ontology of e-portfolios

We use a simple example to illustrate the concept ofontology for e-portfolios. Suppose a course has itsobjectives and rubrics which specify the assessment criteria.The instructor uses various assessment instruments toFig. 1 Primitive ontology

Table 1 Higher-order thinking and support of e-portfolios

Type of higher-orderthinking paradigm

Examples of the higher-orderthinking paradigm

Description of the higher-order thinking paradigm Support of e-portfolios

Non-discipline-specificnon-skills-specific

Career development Think on personal mission, career selection,and long-term goals.

Accumulative learning portfolios,reflection portfolios and assessments

Academic accomplishment Think to plan academic success, and to recognize gapsbetween the existing knowledge and curricula competences.

Learning portfolios collection

Extra-curricular learning Think to celebrate broad life experiences, to develop socialskills and responsibility.

Reflection portfolios

Non-discipline-specificskills-specific

Self-regulation The thinking ability to self-monitor and to learn fromexperiences and mistakes.

Reflection portfolios

Motivation The thinking ability to be effortful and creative. Learning portfolios and assessment

Discipline-specific (e.g.,Business education)

Critical thinking Think on rationale decision making and judgment. Learning portfolios and assessment

Design thinking Think on managerial process plans. Learning portfolios and assessment

Systems thinking Think on diversified elements and factors of systemsand their interconnected relationships.

Learning portfolios and assessment

Inf Syst Front (2012) 14:395–407 399

assess students’ learning, such as examinations, essays, andcourse projects. The students’ part of e-portfolios containsessays, course project reports, and other works. Theinstructor provides evaluations and comments on thestudents’ essays and course projects. The ontology behindthis example is fairly simple. Using the RDF terms, thereare six types of resources for the e-portfolios: course,objective, rubric, assessment instrument, portfolio work,and evaluation and comment. The relationships betweenthese resources can be generic terms such as “Has_a”,“Is_a”, “Uses”, “Applies”, and “Associates”. The abovedescriptions can be briefly formalized into an ontology forreflection, as depicted in Fig. 2.

The ontology in Fig. 2 can be further formalized using acomputer language such as Web Ontology Language (OWL2010) which is designed specifically for procedural appli-cations such as search. Clearly, if the ontology is used todevelop organizational schemata of e-portfolios for instruc-tors and students, all users can share the common teaching/learning scheme. Other advantages of the use of ontology,such as ontology re-use and achievable computer aideddynamic linkages of e-portfolio objects, can be perceivedfrom this simple illustrative example.

5 A case study of e-portfolios schemata for teachingand learning higher-order thinking

5.1 User-driven ontologies of e-portfolios

In the broader literature, there is a lack of formal ontologicaldescription of e-portfolios for higher-order thinking. Further-more, the entire ontology of an e-portfolios repository couldbe very large. To provide a large ontology visual andmanageable to the user, the entire ontology must bepartitioned. This is done through categorizing e-portfolioobjects and developing the dynamic and inheritance relation-ships. Such a formalized generic e-portfolio objects categorycan help a community in developing its ontology, especiallywhen the e-portfolio system is incorporated into the institu-tional learning system.

An ontology represents the concepts commonly sharedby a community (Guarino 1995). Hence, ontologies areuser-driven rather than innate. A user-driven domainanalysis would make e-portfolios a concrete tool forhigher-order thinking instead of historical achieves. Adomain analysis is the identification and formalization offundamental e-portfolio objects and their relationships. Theformalization of those objects and relationships provides abase for the ontological organizations of e-portfolios forhigher-order thinking.

5.2 Overview of the case study

An organizational schema of e-portfolios is a user-computerinterface that allows the user to manipulate e-portfolios guidedby the ontology. It provides visualization of the organizationof e-portfolio objects for easy navigation, and is a “road-map”for the user (student or teacher) to access e-portfolio objects.The schema for teachers are different from that for students inthat the teacher’s schema allows teachers to generate thinkingroutines, while the student’s schema allows students to learnhigher-order thinking.

To learn more about organizational schemata ofe-portfolios for higher-order thinking, a domain analysiswas conducted as described in the next sub-section. Thisstudy uses the business discipline as an example. Toimprove the overall teaching-learning quality of thebusiness education, it is important to engage students inactive thinking. As shown later in this section, the domainanalysis formalizes the schemata of e-portfolio objectswhich may or may not be the direct products of coursesbut could be used for teaching and learning higher-orderthinking. An ontology with a small scale and thecorresponding ontological schemata for higher-order think-ing through the use of e-portfolios was developed. Aprototype of software system, called ThinkOn (Thinkingthrough the Ontology), that was used to support higher-order thinking through the use of e-portfolios for businesswas implemented. This project is used merely to demon-strate the principles of ontological schemata for organizinge-portfolios to foster higher-order thinking discussed above,

Fig. 2 The ontology of theillustrative e-portfolios example

400 Inf Syst Front (2012) 14:395–407

but not for discussion of pedagogical designs of courseswhich is a topic outside of the scope of this study.

5.3 The domain analysis

Domain analysis is a process that identifies a knowledgeorganization for a certain subject area (Hjorland andAlbrechtsen 1995). In the software development field, thetarget outcome of a domain analysis is a set of objectclasses and their interrelationships that can be used forinformation system development (Prieto-Diaz 1990). Do-main analysis is basically an epistemological approach. Ithas not been fully formulated as a theoretical researchapproach in the IS field, although it has been applied in theIS literature (e.g., Wang 1999b). There is no neutralplatform other than experts in the domain that can be usedfor assessing the result of a domain analysis. Also, the userscan be influenced by the result of a domain analysis indetermining information requirements.

The method employed in the present domain analysiswas an analysis of business student e-portfolios. Twentythree (23) high quality business student e-portfolios withreflection components over the past several years werereviewed. The number of observations is not large, but wasadequate given the nature of the analysis used. The sampleincluded two sets. One set with 18 was used directly fordomain analysis. The rest of the e-portfolios (5) were usedfor validation. The domain analysis process reported in(Prieto-Diaz 1990) was applied in this study, as brieflydescribed below.

& Review the sample reflection e-portfolios.& Identify specific objects related to teaching and learning

higher-order thinking.& Abstract the objects, and make classification.& Identify and abstract the relationships between the

classes.& Formalize the classes and their relationships into a

network which is the frame of the ontology to bedeveloped.

& Use validation sample set to check the network.

5.4 General types of e-portfolio objects for higher-orderthinking in business education

In this subsection, we discuss the fundamental types ofe-portfolio objects and their relationships which havebeen identified in our domain analysis.

(1) Thinking paradigm

A thinking paradigm is a higher-order thinking modewhich requires certain cognitive characteristics. As dis-

cussed earlier in this paper, the taxonomy of thinkingparadigm highly depends on unique cognitive aspects. Thedesign and explanation of thinking paradigms for adiscipline is the first task for teachers to foster students’higher-order through the use of e-portfolios.

(2) Model and Thinking Query

Models can be a tool of compelling management higher-order thinking (Dunne and Martin 2006). While theultimate models in great managers’ mind might not beavailable, models taught in business courses provideguidelines for management higher-order thinking. Forinstance, the decision making model (Simon 1976) taughtin the introduction MIS course can help students developcritical thinking dispositions. Students should apply themodel to any managerial decisions in the businessdiscipline and think about the decision making as well asthe important roles of data and information in decisionmaking. A model object has questions, or thinking queries,for students to foster higher-order thinking. For instance,the decision making model can have thinking queries suchas: How is the decision making model related to the caseyou analyzed? Why did the decision in the case youanalyzed succeed or fail in the view of the decision makingmodel? etc. Students’ works can be linked to these queriesfor student to think.

(3) Work

Awork is an academic document that records a student’slearning product such as essay, project report, case analysis,technical assignment, etc. A work object can have compo-nents as its sub-objects, and each component can havemultiple super-objects. To make searching easier, metadatasuch as work type, course, assessor, time, etc., are attachedto a work object.

(4) Reflection

A reflection object is a documentation of students’higher-order thinking outcome. A reflection object isdifferent from a work object in that a reflection objectclearly demonstrates higher-order thinking disposition. Forinstance, a video clip created for a course is a work object,but is not a reflection object unless the student’s higher-order thinking disposition is explicitly demonstrated. Theontological distinction of work and reflection makes thesearch for relevant e-portfolio objects efficient. Clearly, ane-portfolio object can have multi-inheritance relationshipswith work and reflection.

(5) Comment

A comment object is a documentation of feedback of theassessor or a group member for the student’s work. At theelemental level, a comment object can be an individual

Inf Syst Front (2012) 14:395–407 401

note. At the collective level, a summary of comments canbe an object to measure the effectiveness of higher-orderthinking. In the educational literature, the use of commentsto improve students’ higher-order thinking is underreported.This is because verbal comments are hard to process bycomputers. Along with the proliferation of e-portfoliosystems, massive comment objects are stored online. Thesecomment objects provide valuable quantitative and qualita-tive data for higher-order thinking.

(6) Dynamic linkages of the e-portfolio objects

An ontological schema of e-portfolios for thinking is asynthesis of these general categories of objects on thecontingency of knowledge sharing in the higher-orderthinking community as well as individual user’s preferences.The major task of the construction of an ontologicalschema is to identify the models and their thinkingqueries for higher-order thinking, which is presumablydone by the teacher of thinking. The relationshipsbetween these objects could be generic semantics suchas “Has_a”, “Is_a”, “Uses”, “Applies”, and “Associates”.By using the organizational schema, the user (the teacher ofthinking or the student) is allowed to link the object instancesas guided by the ontology. In an object-oriented system, theserelationships can be implemented through inheritance (forstatic relationships such as “Is_a” and “Has_a”) or messagesending (for dynamic relationships such as “Uses” and“Applies”). Next, we present an example of implementationof ontological schemata of e-portfolios for higher-orderthinking for the business discipline.

5.5 Ontological schemata of e-portfolios:ThinkOn prototype

In this sub-section, we demonstrate ontological schemata ofe-portfolios through the ThinkOn prototype. The tool usedto build ThinkOn was Microsoft Excel, because the currentstandard ontology modeling language (OWL) does notsupport user interface which was crucial for this case, andExcel is a popular end-user computing tool. Also, fewontology editors available on the market meet our needs.The internal database (spreadsheets) of ThinkOn storedmetadata of the e-portfolio objects and hyperlinks to theactual e-portfolio objects which were supposed to be storedon the e-portfolio systems and the Internet in general. TheExcel environment provided the visualization of theontological schemata. The command buttons for exploringthe ontological schemata were implemented using ExcelVBA.

Figure 3 shows the ontology of the inter-relationalstructure of e-portfolios for fostering higher-order thinkingat the top level. As shown in the ontology network,thinking paradigm, model, thinking query, work, reflection,

and comment objects were semantically linked and repre-sent the organization of e-portfolio objects for higher-orderthinking.

Two schemata were developed: teacher’s schema andstudent’s schema, which represented the two views of thetwo types of users on the common ontology for their teaching/learning thinking activities. Figure 4 shows the teacher’sschema along with a working pane. Each e-portfolioobject had methods associated with the commandbuttons. The command buttons on the relationshipallowed the teacher to implement the dynamic relation-ships between the e-portfolios for teaching higher-orderthinking. The teacher was allowed to search, add, andlink e-portfolio objects through the schema. The workingpane was used to view the metadata of e-portfolio objectsand to access e-portfolio objects though hyperlinks. As shownin the example in Fig. 4, the teacher was viewing themetadata of a thinking query and a student work, and wasassigning the thinking query to the student work afterclicking on the “Applies” command buttons. In this example,once the teacher applied the thinking query to the student’swork, the state of the student work was changed to “thinkingpending” so that the student would be notified of thethinking assignment.

Figure 5 shows the student’s schema along with aworking pane. The student’s schema typified students’activities in learning higher-order thinking. The student wasallowed to search, add, and link work and reflection objectsthrough the schema. The working pane was used to carryout learning higher-order thinking through the e-portfolios.The example in Fig. 5 shows that the student was having athinking assignment, and was viewing the metadata of athinking query and her own work after clicking on the“Associate” command buttons. Once the student completedher reflection, the state of the student work was changedfrom “thinking pending” to “reflection completed”.

5.6 Pilot test

A pilot test, which was independent of the domain analysisand the development of the prototype, was conducted tovalidate the potential benefits of the ThinkOn. This pilot testwas aimed at comparing traditional e-portfolios and ThinkOnwith respect to the efficiency in managing activities forhigher-order thinking, and the usability of ThinkOn. Tomake a comparable computing environment, the traditionale-portfolios (i.e., e-portfolios with the flat structure) wereemulated by Excel spreadsheets. Each sheet represented anartifact with a simple (no more than 50 words) description.The artifact names were displayed on the sheet tabs.

The pilot test involved two teachers and five students.Both teachers were professors in the MIS field. All studentswere first-year university students in business, three

402 Inf Syst Front (2012) 14:395–407

females and two males. The pilot test was explained to eachsubject individually.

The teachers were asked to assign five thinking queriesto five work objects using the traditional e-portfolios. Thetimes used for the activities were counted. They were thenasked to assign another five thinking queries to another setof five work objects using ThinkOn. It was found thatThinkOn was slightly better than the traditional e-portfoliosystem in terms of time saving, but the difference ofefficiency of the two systems was not statistically signifi-cant. A possible reason was that these tasks were so simpleto the teachers who might already possess the concept oforganizational schemata of e-portfolios.

The students were asked to find five right thinkingqueries to match five work objects using the traditionale-portfolios and ThinkOn, respectively. The times spentby the students to correctly establish the links weremeasured. It was found that ThinkOn significantly

reduced the times for managing the artifacts which wererelative to higher-order thinking.

After completing the trials, each subject was asked tocomplete a usability questionnaire by judging the state-ments in the questionnaire using a 5-point scale, where 1was “strongly disagree” and 5 “strongly agree”. Thequestions and average scores are summarized in Table 2.

While this study makes no claim to a contribution of theThinkOn prototype to the effectiveness of higher-orderthinking, the results from the pilot test does demonstratesome usefulness of such tools for teaching and learninghigher-order thinking. The preliminary pilot test has openedtasks for our future research in addition to improving theprototype. First, a set of criteria for rigorous tests, includingthe impact of e-portfolio schemata on the effectiveness ofhigher-order thinking, must be created. Second, a compre-hensive design of experiment, including the selection ofsubjects, treatments, and procedures, must be developed.

Fig. 4 Teacher’s schema of e-portfolios for fostering higher-order thinking

Fig. 3 The ontology ofe-portfolios for higher-orderthinking

Inf Syst Front (2012) 14:395–407 403

5.7 Lesson learned from the case study

The following primary lessons have been learned from thiscase study.

(1) The ontological schemata of e-portfolios for higher-order thinking provide the semantic associationsamong the e-portfolio objects. It is high-level codifiedknowledge of teaching/learning thinking. Hence, itreflects the educational community's memory.

(2) The ontological schemata provide maps for the user tonavigate the e-portfolios so that he/she is able tosearch semantically closely bounded e-portfolioobjects for higher-order thinking. The schematafacilitate the integration of multiple disciplinaryhigher-order thinking.

(3) To reduce information overload, a schema can bepartitioned into sub-schemata so that each part canappear on the screen for the user. The principle ofpartitioning is that each part of the schema keeps anontological unity.

(4) There might be multiple entries for the user to reach aparticular e-portfolio object through a navigation ofthe entire schema. The design of each sub-schema canbe individualized for different types of users while all

sub-schemata follow the same ontology. For instance,the layout of display could be adjusted, and short-cutscan be created for sophisticated users to access aspecific segment of the e-portfolios.

(5) Ontology is user-driven. Clearly, the schemata wepresented in this case study emulates the way ofteaching higher-order thinking for the business disci-pline as suggested by the investigator of this study.Each user (teacher or student) can create his/her ownontological schemata for higher-order thinking toeliminate baseless free thinking.

(6) Finally, higher-order thinking through ontologicalschemata is by no means at no cost. The first stepfor teaching higher-order thinking is to identify/designthinking paradigms, models, and thinking queries. Toconnect a learning object to the schema, one must linke-portfolio objects (through the methods of theobjects) in accordance with the ontology. The morelinkages are defined, the more paths would beavailable for higher-order thinking. To fully useontological schemata to create a dynamic teachingand learning higher-order thinking environment, onemust implement a variety of methods for the e-portfolios objects. For instance, when a teacher wantsto raise the student’s thinking to the next level, there

Fig. 5 Student’s schema ofe-portfolios for conductinghigher-order thinking

Question Average score

Q1 ThinkOn is easy to learn 3.4

Q2 ThinkOn is useful for linking documents I needed 4.6

Q3 ThinkOn is easy to use 3.8

Q4 ThinkOn makes higher-order thinking more visible 4.2

Q5 ThinkOn makes higher-order thinking more teachable 4.2

Q6 ThinkOn has expected capacities for teaching/learning higher-order thinking 3.8

Q7 Overall, ThinkOn is a good system for teaching/learning higher-order thinking 4.2

Table 2 The usabilityquestionnaire

404 Inf Syst Front (2012) 14:395–407

must be methods for certain actions that the teacher aswell as the student would undertake to facilitate thehigher-order thinking. This is the nature of teachingand learning through the use of e-portfolios.

To thoroughly test a design of information systemartifacts such as e-portfolio schemata for higher-orderthinking, rigorous independent experiments must be con-ducted. This case study has its limitation in that theontological model has not reached practical trials beyondthe design experience. Clearly, while this study makes noclaim to the validity of the proposed model, it does offer apragmatic instrument that drives e-portfolios towardssupporting high-order thinking, as ThinkOn has demon-strated. This study has made its initial contribution to theaccumulated weight of empirical evidence for establishingthe validity of this ontological methodology.

6 Conclusion

The competence of e-portfolios depends not only on theabundance of artifacts, but also the effectiveness of the use ofe-portfolios in reflection and active thinking. This paperrecognizes the problem of lack of organizational models for e-portfolios, and proposes a framework of ontological schematafor e-portfolio objects repositories. The ontological schemamodel is based the premise that higher-order thinking isteachable. The contribution of this study is the conceptualontological model that can be used for the construction ofhigher-order thinking oriented e-portfolios schemata. Anontology adds explicit relationships between the e-portfolioobjects that would aid the user to conduct desired higher-orderthinking. The use of ontologies would allow the user tovisualize thinking. An ontology can be a teacher’s teachingtool for teaching higher-order thinking, or student’s owndesigned instrument for reflection and active thinking.

Technically, this study has primarily focused on thesemantic aspects of e-portfolios for higher-order thinking,and has shown the support of semantic Web techniques fore-learning. It raises new challenges for all parties involvedin e-learning. For educational institutions, there is anorganizational need to develop ontologies that containsemantic information about the higher-order thinking invarious paradigms. The ontologies should be maintainableto represent the currency of e-portfolios. For softwaredevelopers, new techniques and tools are imperative todevelop comprehensive uses of e-portfolios beyond assess-ment and showcase. Simple documentations of academicrecords are no longer adequate. In our view, the ontologicalmodel proposed here can practically be used for e-portfoliosoftware. For teachers, new skills of teaching higher-orderthinking are required. They must clearly understand the

ontological structure of e-portfolios for higher-order think-ing, and transform unstructured higher-order thinkingactivities to structured tasks based on their own ontolo-gies on higher-order thinking. For students, applicationsof e-portfolios for higher-thinking will be a new opportunityof e-learning. Our experiences with this study demonstratethat the model of organizational schemata of e-portfolios canhelp students meet the new challenge.

Acknowledgement The comments of two anonymous reviewershave contributed significantly to the revision of this paper.

References

AACSB. (2003). Eligibility procedures and standards for businessaccreditation, Association to Advance Collegiate Schools ofBusiness (AACSB) International, [Retrieved March 14, 2010from http://www.aacsb.org].

Annis (Ferrill), L., & Jones, C. (1995). Student portfolios: Theirobjectives, development, and use. In P. Seldin & Associates (Eds.),Improving college teaching (pp. 181–190). Boston: Anker.

Argyris, C., & Schon, D. A. (1988). Organizational learning. In D. S.Pugh (Ed.), Organization theory selected readings (pp. 352–370).New York: Penguin.

Barolli, L., Koyama, A., Durresi, A., &DeMarco, G. (2006). Aweb-basede-learning system for increasing study efficiency of stimulatinglearner’s motivation. Information Systems Frontiers, 8, 297–306.

Barrett, H. C. (1994). Technology-supported assessment portfolios.The Computing Teacher, 21, 9–12.

Batterbee, L., & Dunham, A. (2004). Four years of reflection: Thedigital portfolio project at Albion College. In J. Zubizarreta (Ed.),The learning portfolio: Reflective practice for improving studentlearning (pp. 59–63). San Francisco: Jossey-Bass.

Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: Theclassification of educational goals, Handbook I: Cognitivedomain. Green: Longmans.

Boland, R. J., & Collopy, F. (Eds.). (2004). Managing as designing.Stanford: Stanford University Press.

Bruder, I. (1993). Alternative assessment: Putting technology to thetest. Electronic Learning, 12, 12–28.

Bushweller, K. (1995). The high-tech portfolio. The ExecutiveEducator, 17, 19–22.

Carchiolo, V., Longheu, A., Malgeri, M., & Mangioni, G. (2007). Amodel for a web-based learning system. Information SystemsFrontiers, 9, 267–282.

Cohen, E., & Nycz, M. (2006). Learning objects e-learning: Aninforming science perspective. Interdisciplinary Journal ofKnowledge and Learning Objects, 2, 23–24.

Collis, B., & Strijker, A. (2003). Re-usable learning objects in context.International Journal on E-Learning, 2(4), 5–16.

CW (2010). Chalk & Wire, [Retrieved March 14, 2010 from http://www.chalkandwire.com/eportfolio].

DC (2010). Dublin core metadata initiative, [Retrieved March 14,2010 from http://dublincore.org].

Devedzic, V. (2004). Education and the semantic web. InternationalJournal of Artificial Intelligence in Education, 14, 39–65.

Drier, H. N. (1997). Career portfolios — don’t leave home without one.Career Planning & Adult Development Journal, 12(4), 55–60.

Dunne, D., & Martin, R. (2006). Design thinking and how it willchange management education: An interview and discussion.Academy of Management Learning & Education, 5(4), 512–523.

Inf Syst Front (2012) 14:395–407 405

Eliot, J. (1987). Models of psychological space: Psychometric,developmental, and experimental approaches. New York:Springer-Verlag.

Ericsson, K. A., & Smith, J. (1991). Toward a general theory ofexpertise: Prospects and limits. Cambridge: Cambridge Univer-sity Press.

Eysenck, M. W. (1993). Principles of cognitive psychology. Hillsdale:Lawrence Erlbaum.

foliotek (2010). foliotek e-portfolio system, [Retrieved March 5, 2010from http://www.foliotek.com/].

Greenberg, G. (2004). The digital convergence: Extending theportfolio. Educause Review, 39(4), 28–36.

Gregor, S. (2006). The nature of theory in information systems. MISQuarterly, 30(3), 611–635.

Gruber, T. (1993). A translation approach to portable ontologyspecifications. Knowledge Acquisition, 5(2), 199–220.

Gruber, T. (1995). Toward principles for the design of ontologies usedfor knowledge sharing. International Journal of Human Com-puter Studies, 43(5/6), 907–928.

Guarino, N. (1995). Formal ontology, conceptual analysis andknowledge representation. International Journal of HumanComputer Studies, 43(5/6), 625–640.

Harman, K., & Koohang, A. (2005). Discussion board: A learningobject. Interdisciplinary Journal of Knowledge and LearningObjects, 1, 67–77.

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science ininformation system research. MIS Quarterly, 28(1), 75–105.

Hjorland, B., & Albrechtsen, H. (1995). Toward a new horizon ininformation science: Domain analysis. Journal of the AmericanSociety for Information Science, 46(6), 400–425.

IEEE LTSC (2010). IEEE learning technology standards committee.[Retrieved March 15, 2010 from http://ieeeltsc.org].

IMS (2006). IMS Meta-data Best Practice Guide for IEE 1484.12.1-2002 Standard for Learning Object Metadata Version 1.3 FinalSpecification. [Retrieved March 13, 2010, from http://www.imsproject.org/metadata/].

Jenkins, E. K. (1998). The significant role of critical thinking inpredicting auditing student’s performance. Journal of Educationfor Business, 73, 274–279.

Jones, D., & Gregor, S. (2007). The anatomy of a design theory. Journalof the Association for Information Systems, 8(5), 312–335.

KEEP (2010). KEEP toolkit, [Retrieved March 15, 2010 from http://www.cfkeep.org/static/index.html].

Kida, T. (2006). Don’t believe everything you think: The 6 basicmistakes we make in thinking. Amherst: Prometheus Books.

Kim, D. H. (1993). The link between individual and organizationallearning. Sloan Management Review, 35(1), 37–50.

Kirkwood, M. (2000). Infusing higher-order thinking and learning tolearn into content instruction: A case study of secondarycomputing studies in Scotland. Journal of Curriculum Studies,32(4), 509–535.

Koohang, A. (2004). Creating learning objects in collaborative e-learning settings. Issues in Information Systems, 4(2), 584–590.

Kurfiss, J. G. (1988). Critical thinking: Theory, research, practice, andpossibilities. ASHE-ERIC Higher Education Report No.2, ASHE,Washington, DC.

Lumsden, J. A., Garis, J. W., Reardon, R. C., Unger, M. P., & Arkin,S. (2001). A blueprint for building an online career portfolio.Journal of Career Planning and Employment, 62(1), 33–38.

McCowan, C., Harper, W., & Hauville, K. (2005). Student e-portfolio:The successful implementation of an e-portfolio across a majorAustralian university. Australian Journal of Career Development,14(2), 40–52.

Mustaro, P. N., & Silveira, I. F. (2006). Learning objects: Adaptiveretrieval through learning styles. Interdisciplinary Journal ofKnowledge and Learning Objects, 2, 35–46.

Myers, I. B. (1962). Introduction to type: A description of the theoryand applications of the Myers-Briggs type indicator. Palo Alto:Consulting Psychologists Press.

Namuth, D., Fritz, S., King, J., & Boren, A. (2005). Principles ofsustainable learning object libraries. Interdisciplinary Journal ofKnowledge and Learning Objects, 1, 181–196.

OSP (2005). Open-Source Portfolio, White paper: UnderstandingOSP. [Retrieved March 15, 2010 from http://www.rsmart.com/assets/understandingOSP_Dec2005.pdf].

OWL (2010). Web Ontology Language (OWL), [Retrieved March 12,2010 from http://www.w3.org/TR/owl-features/].

Page, D., & Mukherjee, A. (2007). Promoting critical-thinking skillsby using negotiation exercises. Journal of Education forBusiness, 82, 251–257.

Peach, B. E., Mukherjee, A., & Hornyak, M. (2007). Assessingcritical thinking: A college’s journey and lessons learned.Journal of Education for Business, 82, 313–320.

Peirce, C. S. P. (1997). Pragmatism as a principle and method of rightthinking. In P. A. Turrisi (Ed.), The 1903 Harvard lectures onpragmatism. Albany: State University of New York Press.

Perkins, D., Jay, E., & Tishman, S. (1993a). Introduction: Newconceptions of thinking. Educational Psychologist, 28(1), 1–5.

Perkins, D., Jay, E., & Tishman, S. (1993b). New conceptions ofthinking: From ontology to education. Educational Psychologist,28(1), 67–85.

Plous, S. (1993). The psychology of judgment and decision making.New York: McGraw-Hill.

Prieto-Diaz, R. (1990). Domain analysis: An introduction. ACMSIGSOFT Software Engineering Notes, 15(2), 47–54.

Quillian, R. (1968). Semantic memory. In M. Minsky (Ed.), Semanticinformation processing. Cambridge: MIT Press.

Ritchhart, R. (2002). Intellectual character: What it is, why it matters,and how to get it. San Francisco: Jossey Bass.

Senge, P. M. (1990). The fifth discipline: The art and practice of thelearning organization. New York: Currency/Doubleday.

Senge, P. M. (1996). Systems thinking. Executive Excellence, 13(1),15–16.

Shen, L., Callaghan, V., & Shen, R. (2008). Affective e-learning inresidential and pervasive computing environments. InformationSystems Frontiers, 10(4), 461–472.

Sicilia, M. A., & Lytras, M. (2005). On the representation of changeaccording to different ontologies of learning. InternationalJournal of Learning and Change, 1(1), 66–79.

Simon, H. A. (1976). Administrative behavior (3rd ed.). New York:The Free Press.

Simon, H. A. (1996). The sciences of the artificial (3rd ed.).Cambridge: The MIT Press.

Singh, G., Hawkins, L., & Whymark, G. (2007). An integrated modelof collaborative knowledge building. Interdisciplinary Journal ofKnowledge and Learning Objects, 3, 85–105.

Smrz, P. (2004). Integrating ontologies into learning managementsystems — A case of Czech. In R. Meersman, Z. Tari & A.Corsaro, OTM Workshops, LNCS 3292, 768–772.

Snae, C., & Brueckner, M. (2007). Ontology-driven e-learning systembased on roles and activities for Thai learning environment.Interdisciplinary Journal of Knowledge and Learning Objects, 3,1–17.

TaskStream (2010). TaskStream e-portfolio system, [Retrieved March12, 2010 from https://www.taskstream.com/pub/].

Thurston, E. K. (2000). Enabling systems thinking in the “MesonicMillennium”: The need for systemic methodologies for concep-tual learning in undergraduate management education. Journal ofManagement Education, 24(1), 10–31.

Walls, J. G., Widmeyer, G. R., & El Sawy, O. A. (1992). Building aninformation design theory for vigilant EIS. Information SystemsResearch, 3(1), 36–59.

406 Inf Syst Front (2012) 14:395–407

Wang, S. (1999a). Analyzing business information systems: An object-oriented approach. Boca Raton: CRC Press.

Wang, S. (1999b). Organizational memory information system: Adomain analysis in the object-oriented paradigm. InformationResources Management Journal, 12(2), 26–35.

Wang, S. (2008). Ontology of learning objects repository foreducational knowledge sharing. Interdisciplinary Journal ofKnowledge and Learning Objects, 4, 1–12.

Warnick, B., & Inch, E. S. (1994). Critical thinking and communication(2nd ed.). New York: Macmillan.

Wiley, D. A. (2000). Learning object design and sequencing theory,Unpublished PhD Thesis, Brigham Young University, 2000.

Wiley, D. A., & Edwards, E. K. (2002). Online self-organizing socialsystems: The decentralized future of online learning. QuarterlyReview of Distance Education, 3(1), 33–46.

W3C (2010). World Wide Web Consortium, [Retrieved March 18,2010 from http://www.w3.org/].

Zhang, D., & Nunamaker, J. F. (2003). Powering e-learning in the newmillennium: An overview of e-learning and enabling technology.Information Systems Frontiers, 5(2), 207–218.

Zhang, S. X., Olfman, L., & Rectham, P. (2007). Designing eportfolio2.0: Integrating and coordinating web 2.0 services with eportfoliosystems for enhancing users’ learning. Journal of InformationSystems Education, 18(2), 203–214.

Zouaq, A., Nkambou, R., & Frasson, C. (2007). An integratedapproach for automatic aggregation of learning knowledgeobjects. Interdisciplinary Journal of Knowledge and LearningObjects, 3, 135–162.

Zubizarreta, J. (2004). The learning portfolio: Reflective practice forimproving student learning. San Francisco: Jossey-Bass.

Shouhong Wang is a Professor of Management Information Systemsat University of Massachusetts Dartmouth. He received his Ph.D.(1990) in Information Systems from McMaster University. Hisresearch interests include IT management, data mining, and teachinghigher-order thinking. He has published several books and over 90papers in academic journals including Journal of ManagementInformation Systems, Information & Management, InternationalJournal of Information Management, Information Resources Manage-ment Journal, Information Systems Management, Journal of Elec-tronic Commerce Research, IEEE Transactions on Systems, Man, andCybernetics, Management Science, Decision Sciences, Journal ofInformation Systems Education, and others.

Hai Wang is an Associate Professor in the Department of Finance,Information Systems, and Management Science at Sobey School ofBusiness of Saint Mary’s University. He received his Ph.D. inComputer Science from the University of Toronto. His research andteaching interests are in the areas of database management andknowledge management. His papers have been published in VLDBJournal, Performance Evaluation, Journal of the Operational Re-search Society, Managerial and Decision Economics, Journal ofInformation Systems Education, and others.

Inf Syst Front (2012) 14:395–407 407