1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  60  61  62  63  64  65  

Towards Trustworthy Smart Cyber-Physical-SocialSystems in The Era of Internet of Things

Jingwei Huang, Mamadou D. Seck, and Adrian GheorgheDepartment of Engineering Management and Systems Engineering

Old Dominion University2101 Engineering Systems Building, Norfolk, VA 23529

Email: {j2huang,mseck,agheorgh}@odu.edu

Abstract—The advent of a new wave of computing drivenby the Internet of Things (IoT) and Big Data is reshapingthe landscape of engineering systems design, operations, andmanagement. As traditional devices and systems are transforminginto smart devices and smart cyber-physical systems poweredby IoT, more and more applications of Internet of Things areemerging as complex systems of smart systems, such as smartcities, in which an entity (including both human, software, andmachines) highly relies on other entities in a network withrespect to security, privacy, trustworthiness of data / information,and trustworthiness of services. Trust is emerging as a criticalfactor in systems design, operations and management. In thispaper, we use Smart and Connected Senior Caring Systems asdriving application, to discuss the features of Cyber-Physical-Social Smart Systems (CPS3) powered by IoT, to address theneeds of trust in CPS3 design, and to explore approaches oftrust formalisms for CPS3 design.

Keywords: Internet of Things; Cyber-Physical Systems;Cyber-Physical-Social Systems; System of Smart Systems;System of Systems; Trustworthiness; Trust.

I. INTRODUCTION

The great wave of computing driven by the Internet isstill peaking; now we are on the path to yet another greatwave, driven by Internet of Things (IoT), together with mobilecommunication and computing, cloud computing and BigData. IoT, which connects things in the physical world with thecyber world, will be a trigger of many innovative applications.In a recent US National Intelligence Council report, IoTwas identified as one of six disruptive civil technologies [1].Indeed, through sensors and actuators, smart things in thephysical world can be remotely located, monitored, and evencontrolled through the Internet; with mobile devices, peoplewill have ubiquitous access to the Internet and the connectedsmart things in the physical world as well as the cloud servicesneeded to handle those smart things. IoT, together with cloudcomputing, mobile communication & computing, semanticweb, and social computing, will lead to many novel Cyber-Physical-Social Smart Systems (CPS3), such as smart cities,smart parking, smart traffic, smart homes, smart healthcare,smart transportation, smart supply chains, smart manufactur-ing, smart product life cycle management, smart environmentmonitoring, smart government services, ..., and so on. We willlive in an integrated cyber-physical smart planet! On the otherhand, we will face broader and greater challenges on security,privacy and trust. Correspondingly, in the engineering world,

there will be a fundamental transformation in engineeringsystems design, operations and management.

Behind those technological advances, societal changesand global issues such as globalization, urbanization, eco-environment and climate changes, sustainability, and agingare stimulating new needs for technologies, products andservice. Those challenges provide fertile fields for the growthof novel cyber-physical-social smart systems powered by IoT.The conjunction of the societal challenges and the technologyinnovations triggered by IoT will breed new classes of en-gineered systems whose design, operations, and managementwill pose great challenges to systems engineers. Indeed, thesesystems, typically a system of smart systems such as smartcities, will consist of dynamic networks of devices, softwareand people, orchestrated in a complex and decentralized way,to generate new system services, some of which will be highlycritical smart services.

These envisioned systems share many novel characteristics.They entangle cyber, physical and social components (sensors,apps and people collaborate to deliver a unique service); theyare diffuse (where exactly is your data when you put it in thecloud?); they continuously reconfigure (nodes enter and exitsystem seamlessly, – think of Uber); finally, they are intimate(Fitbit monitors your heart rate and archives it on the cloud).

On account of these new characteristics, traditional qualitiessuch as safety, reliability, or usability, which were exclusivelycentered on the technical system itself, are not anymoresufficient. Other qualities are needed to account for the com-plex relation that holds between the social and the technical.Chief among these qualities is the notion of trust. In theprocess of designing these Cyber-Physical-Social Systems, thedesigner will not only have to ask herself how to architectand design the system so that it is safe or reliable, but also,how to architect and design the system so that it is deemedtrustworthy. This is particularly important because a systemthat is not trusted will either not be used at all, or will be usedin ways that subvert its initial intent. For example, considerhow the lack of trust in the privacy of a health monitoringservice could lead a user to tamper with a sensor temporarily,thereby causing the system to misdiagnose a serious condition.

In this emerging integrated cyber-physical smart world, oneentity heavily relies on other entities in a network with respectto security, privacy, trustworthiness of data / information, and

SoSE 2016 1570255165

1

trustworthiness of services. When an entity needs to trustanother, this “trust” frequently becomes a vulnerability. Tomitigate the vulnerability due to trust in engineering systemsdesign, operations, and management, we need to formalize“trust”, and handle trust in a rigorous way. We begin witha motivating example – Smart and Connected Senior CaringSystems, in the next section, and discuss the features of thisnew type of systems – CPS3; then we discuss the needs ofconsidering trust in systems design in section 3; then in section4, we discuss the framework we are working on towardsdeveloping a computational theory of trust for CPS3 design;we summarize our discussion in section 5.

II. MOTIVATING APPLICATION: SMART AND CONNECTEDSENIOR CARING SYSTEMS

A. Societal needs from an ageing world

According to UN World Population Ageing report (2013)[2], population ageing is occurring world wide; the proportionof old persons (60+) globally was 9.2% in 1990 and 11.7% in2013, and is projected to be 21.1% in 2050; this populationageing is unprecedented, given that by 2050 the number ofolder persons (60+) will exceed the number of the young (15-)in history; the old-age support ratio, the number of working-age persons (between 15 and 64) per old person (65+) in theworld, was 12 in 1950 and 8 in 2013, and is projected to fall to4 in 2050; the old-age support ratio in developed countries willdrop from 4 in 2013 to 2 in 2050. this population ageing hasbroad and profound impacts to almost every aspects of humanlife, such as health, healthcare and healthcare facilities, familycomposition and living arrangement, saving, investment andconsumption, labour market, pensions, tax, intergenerationalfairness, and even voting patterns and representation.

B. Smart and Connected Senior Caring Systems (SCSCS)

The goals of SCSCS are to efficiently and effectivelyprovide a more comfortable living and caring environment forsenior people and to reduce the increasing pressure caused byageing for families and societies, by leveraging the technologyprogress on Internet of Things, Smart Homes, Telecommu-nication, Cloud Computing, and Big Data, by synthesizingthose technologies with social computing, social networks, andprofessional services in the real world to form a new type ofCyber-Physical-Social Smart Systems.

In this envisioned system, an elderly person can subscribe toa personalized package of care services through a senior careservice provider. These services include: health monitoringservices, to continuously capture vital signs to detect any needfor medical attention and helps; nursing services, with a nursevisiting the elderly person regularly for routine health activ-ities, as well as situation-driven on demand nursing servicessuch as out-patient medical caring; home security services,with sensors that that monitor the home to make sure that thedoors and windows are locked and actuators to take measuresremotely if necessary; home automation, such as appliancecontrol, lighting control, home climate control, leak detection,smoke and CO detection; reminder systems for reminding

everyday activities such as locking doors, turning off the stove,taking medicines, and so on; house-keeping services to offeron demand cleaning, grocery or cooking services.

C. Features of SCSCS

This SCSCS is a representative Cyber-Physical-SocialSmart System (CPS3). It contains cyber components, such asthe algorithms that monitor the health data and cloud servicesfor storing and analyzing health data, physical componentssuch as the sensors and actuators in the home security system,and social components, such as the nurses, doctors, andhouse-keeping personnel. All those components need to betrustworthy to make SCSCS trustworthy.

The SCSCS is a “system of smart systems” in whichmuch of the service functionality is provided by informationtechnology and automation. Sensors are trusted to capturerelevant data, algorithms are trusted to support smart decisions,and actuators are trusted to bring these decisions into action.As such, the system produces large amounts of data from thedigital footprints of the users and the involved sub-systems.The processing and storage of the data is Cloud facilitated,posing special challenges for privacy and security.

Furthermore, the system is decentralized. While the sub-scription to the service package can be done through a uniqueinterlocutor, the services are provided by components whichare not co-located in the same physical place and which arenot operating under a unique organizational boundary. Giventhe prevalent business trend of companies focusing on corecompetencies, it is likely that each portion of the service willbe provided by a distinct organizational entity.

Finally, a key distinguishing feature is that the envisionedsystem is human-centered. As recipient of the service, theelderly person is ultimately the stakeholder who defines thevalue system and assesses the system effectiveness in a sub-jective way, according to his or her perception of performance.Critically, this assessment is affecting the way in which thesystem is used and how it ultimately performs.

These features conjointly create an unprecedented level ofcomplexity for systems design.

III. BRINGING TRUST INTO CPS3 DESIGN: CHALLENGES

Once the importance of trust has been established forCyber-Physical-Social Smart Systems (CPS3), an advocate ofdesigning such systems “for trust” must face the questionof conceptualizing, operationalizing and quantifying trust, sothat alternative designs may be rationally analyzed from theperspective of trustworthiness. What, thus, is the concept oftrust as it pertains to systems design?

A user interacting with a system (or more generally a sub-system interacting with another subsystem) for the provisionof a service has normative expectations about the quality ofthat service. Such expectations roughly correspond with userrequirements, both functional and non-functional. Based onthe lived experience, or sometimes the reported experience onthe performance of a system component, the user will come toform certain beliefs about the quality of the service that will

2

actually be delivered. Based on this belief, the user will (orwill not) be willing to act as if the service will be performedaccording to his or her normative expectation. That is theessence of trust. Trusting a system is to believe and to act as ifthe system will perform as normatively expected, even thoughthe user has little control over the drivers of systems behavior.Thus, trust can be seen as a risk management mechanismat the individual level that confronts normative expectationsand predictive beliefs about system performance. Ill conceived“trust” frequently becomes a vulnerability for the trustor.

As a consequence, mistrust of a system may come from atleast two aspects, namely, the normative expectations aboutthe quality of a service, and the formation of beliefs about thequality of the service. To address these two sources of systemmistrust from a designer’s perspective, we may ask: “how dowe formulate system requirements such that they accuratelyand realistically transcribe the normative expectations of auser?”. Secondly we may ask, “how to affect the processof belief formation about system performance (e.g. throughtransparency or social network mechanisms) such that thebeliefs about system performance be more accurate?”. Aneven more ambitious question would be: “how to provideagency (means to affect system performance e.g. requestprioritization) to the user, so that the belief about systemperformance is more easily aligned with the user’s normativeexpectation?”

Unlike traditional quality attributes, which are system prop-erties exclusive to the system, trust can be operationalized asa relational property that holds between a service recipient(the trustor) and a service provider (a trustee). This relationalproperty applies to the specific set of traditional attributes thatare in salient in a particular context. For example, if a useris determining whether to drive or take a bus to an urgentmeeting, a trust calculation is conducted with a set of qualityattributes such as punctuality and accessibility, to determinewhether the highway or the bus service must be trusted morein this particular context, based on normative expectations andbeliefs about the service.

Thus, trust is not just another quality attribute, it operatesas a way to aggregate and assess system quality attributes ina specific context of use. The normative expectation and thepredictive beliefs about system performance may differ for thesame system in different use cases. Therefore, the formulationof trust can be a way of refining system quality attributes ina context dependent way and a way of aggregating qualityattributes as is relevant to a particular context of system use.

Just as system architects know that “security is achievableby locating critical assets in the deeper layers of a layeredarchitecture”, or that “performance is enhanced by the use ofa few centralized components with limited communicationsbetween them” , we should also know what kinds of structuraland communicational patterns in human-centered engineeredsystems promote trustworthiness: that is the ultimate aim ofthis research. To get there, however, the important prerequisitesare (1) to have a formal understanding of trust, (2) a calculusthat allows us to quantify trust, (3) heuristics that links trust to

system synthesis. In particular, we will construct an EpistemicModal Logic axiomatization for trust, which formally definesthe semantics of trust, reveals important properties of trust,and can be used to explain and model how trust propagates ina network.

IV. MODELING TRUST AND TRUSTWORTHINESS IN CPS3

A. Computational Trust

The concept of trust has been intensively used in theInternet-based systems and applications; many computationaltrust models have been studied intensively and extensively [3],[4], [5], [6]. The well known “BAN” logic [7], developed byBurrows, Abadi and Needham, is among the first to introducetrust in computing. In their work, the concept of trust wasused to infer an agent’s belief from another trusted agent’sbelief about the association of a public key with an identity.The mechanism of trusted third party was introduced in X.509public key infrastructures (PKIs) [8]; a peer to peer decentral-ized trust mechanism, called “web of trust”, was invented inPGP [9]. Marsh [10], quantifies trust among agents in inter-action, based on a thorough examination of the concepts oftrust developed in social sciences. From network perspective,Kleinberg [11] proposed a model using eigenvector to discover“authorities” and “hubs” in a network. This work has profoundinfluence in the field. With the similar idea, Google created itsprestige PageRank algorithm [12], and EigenTrust model [13]was developed for measuring reputation in P2P networks. Inabout two decades, a considerable number of computationaltrust models have been proposed. From the perspective of theapproaches to trust, models can be categorized as: (1) directinteraction experience based [10], [14]; (2) credential basedand policy based [15], [16]; (3) reputation-based [12], [13],[17], [18], [19], [20], which are first developed in informationretrieval and eCommerce, and then extended to more generalapplications; (4) recommendation based and social networksbased [21], [22], [23], [4], [24], [25]; There also exist specificdomain application-driven models, including PKI trust [8],information retrieval [12], P2P networks [13], [26], mobileadhoc networks [6], web services, grid computing, and cloudcomputing [27], [28], [29], and trust management for Internetof Things [30], [31], [32], [33].

B. Trust conceptualization

Trust is a complex social phenomenon. The concepts devel-oped in the social sciences provide an important foundation fortrust formalization. A large body of research has contributed tothe evolution of the conceptualization of trust. In his pioneer-ing research on trust [34], Deutsch defined trust as confidenceon a trusting choice in a situation where the outcome dependson the trustee. He identified different types of trust, includingtrust based upon “confidence”, “innocence”, “impulsivity”,“gambling”, “despair”, “conformity”, “virtue”, and “faith”.Rotter [35] defined “interpersonal trust” as “an expectancyheld by an individual or a group that the word, promise,verbal or written statement of another individual or groupcan be relied on.” Luhmann [36] defined trust as “confidence

3

in one’s expectations”. He addressed that “familiarity is theprecondition for trust”. Beyond interpersonal trust, Luhmannidentified “system trust” – the trust placed in the function ofa system in a society. Many researchers recognized that trustis associated with risk. For example, Gambetta [37] addressedthe idea that trust is fragile due to the limits of knowledge andforesight, and the uncertainty of the behaviors of the trustedagent(s). Mayer et al. [38] further incorporated risk factors intothe definition of trust. He defined trust as “the willingness ofa party to be vulnerable to the actions of another party basedon the expectation that the other will perform a particularaction important to the trustor, irrespective of the ability tomonitor or control that other party.” There are many otherviews of trust. From the perspective of economists, trust is“implicit contracting” [39]. Lewis and Weigert [40] arguedthat trust in reality has emotional and cognitive contents.Blomqvist suggested that trust has two aspects of expectation:“competency (i.e. technical capability, skills and know-how)”and “goodwill which implies moral responsibility and positiveintentions towards the other” [41].

Based on social studies of trust discussed above, our work-ing definition of trust is: Trust is a mental state comprising: (1)expectancy - the trustor expects a specific behavior from thetrustee (such as providing valid information or effectively per-forming cooperative actions); (2) belief - the trustor believesthat the expected behavior occurs, based on the evidence of thetrustee’s competence, integrity, and goodwill; (3) willingnessto take risk - the trustor is willing to take risk for that belief.

According to the types of expectancy in trust, we identifytwo types of trust: trust in performance and trust in belief .The former is trust in what the trustee performs; the latteris trust in what the trustee believes. These two types of trustplay important roles in our trust modeling. Our previous workon logical formalism of trust [42], [43] reveals that logically,trust-in-belief relation is transitive; trust-in-performance rela-tion is not, but can propagate through trust-in-belief relation ina trust network. This explains how and why trust propagatesin social networks.

C. Developing A Calculus of Trust for CPS3 Design

To consider trust /trustworthiness in Cyber-Physical-SocialSmart Systems (CPS3) design, we need to quantify and reasonabout trust/trustworthiness. Based on our previous work [25],[44], we will continue to develop a formal-semantics basedcalculus of trust for meeting the needs of CPS3 design.

1) Trust Quantification and Reasoning: Trust has two typesof uncertainties: (i) randomness, coming from a trustee’s un-predictable behavior; (ii) incompleteness, which is uncertaintydue to a trustor lacking sufficient knowledge or informationto make a judgment. For this reason, we quantify trust withbelief triple < α, β, γ > [45], which represents the probabilitydistribution over three truth values – true, false, and unknown.The belief triple can be interpreted equivalently as the proba-bility of a belief being true is uncertain and within an interval[α, α+ γ][44].

Based on the connections between probability and con-ditionals [46] studied in philosophical logic, the degree oftrust is defined as α = pr(Biϕ|Bjϕ) for trust-in-belief, andα = pr(Biτ(p)|madej(p)) for trust-in-performance, whereBi is a modal logic operator for belief [47], read as agenti believes; ϕ is a proposition, expected to be true in a trustrelation; τ(.) is a predicate transforming a sentence object pinto a proposition; finally, the calculation is meaningful onlyin the condition that the context of trust is true, thus omitted;the degree of distrust is defined as β = pr(Bi¬ϕ|Bjϕ) fortrust-in-belief, and β = pr(Bi¬τ(p)|madej(p)) for trust-in-performance; the degree of untrust γ = 1 − α − β. Trustmetric < α, β, γ > is measured as α = p/m, β = n/m,and γ = u/m, where m is the total number of encounters(interactions on an expectation) between trustor and trusteein the sample space; p is the number of trustor’s positiveexperiences; n is the number of trustor’s negative experiences;u is the number of encounters whose outcomes are unknown;p+ n+ u = m.

Based on the formal semantics of trust and the propertythat trust is propagatable, derived from our epistemic logic oftrust, we can calculate sequence trust aggregation and paralleltrust aggregation as follows. Assume < α1, β1, γ1 > is ametric for a trust-in-belief relation from agent i to j w.r.t.ϕ; < α2, β2, γ2 > is a metric for a trust relation (eithertrust-in-belief or trust-in-performance ) from j to k, then thederived trust relation from i to k (the type is the same asthe second trust relation) < α, β, γ > can be proved as:α = α1 · α2 + β1 · β2; β = α1 · β2 + β1 · α2. An importantproperty of sequence aggregation is that trust degree decreasesexponentially with the length of the chain of trust.

For parallel trust paths, we again applied to the frequencyinterpretation of probability to derive a trust aggregation forparallel entities. This has the form of a weighted average oftrust (distrust) degrees in multiple trust paths, where the weightof each path is proportional to the number of samples used bythe neighbor of the subject in that path [25]. Our preliminarywork shows multiple independent trust paths significantlyincrease trust and certainty. Application of this network-basedtrust reasoning to PKI can be found in [25].

2) Evidence-based Trust Reasoning: We are going to de-velop methods and models of trust reasoning from evidence.For developing evidence-based trust reasoning, we select thefollowing three categories of relatively observable evidence,called CIA triad of trust evidence, in parallel to securityCIA triad. Consistency(C), is about the trustee’s compliancewith some acceptable policies and industrial standards, andalso include the historical performance of the trustee withrespect to the trustor’s expectation. We use “consistency” toreflect the broader concept of “integrity”. The latter conceptrefers to that trustee adheres to the principles acceptable totrustee [38], in general context of societies, cultures, andreligions. Intention(I), is about the trustee’s motivation, goals,and plans. Intention is subsumed by goodwill; the latter isbroader, including invisible moral responsibility to trustor andpositive orientation toward the trustor or the representative

4

Fig. 1. Evidence-based trust reasoning of beliefs in expected items from evidence.

group of the trustor; Ability(A), is about a trustee’s technicaland organizational competence with respect to fulfill a specificexpectation. Our initial idea can be illustrated as shown infigure 1, which shows a framework to infer the trust placed ina cloud service provider w.r.t. privacy protection, from piecesof evidence collected from evidence space including what theservice provider does to gain trust and what users observe.

D. Systems trustworthiness

Several definitions of “trustworthiness” exist in the liter-ature, such as, “does what people expect it to do and notsomething else” [48]; “assurance that a system will perform asexpected” [49]. What people expect may cover a broad rangeof attributes, such as availability, reliability, maintainability,safety, integrity, confidentiality, privacy, and resilience. Thoseattributes are beyond the range covered by dependabilityand security as defined in [49]. The attributes of interestmay shift for different stakeholders and different contexts.Trustworthiness reflects collective trust in a set of systemattributes. Technically, we regard trustworthiness as beliefs inexpected attributes. Those beliefs can be derived from relationsamong those attributes.

We adopt an approach to modeling system trustworthinessin Belief Networks, so that evidence-based trust reasoning andprobabilistic causal reasoning can be easily integrated.

V. CONCLUDING REMARKS

This paper has highlighted the importance of trust in thedesign of Cyber-Physical-Social Smart Systems powered bythe Internet of Things. We consider trust as a critical prop-erty that operates between the Cyber-Physical and the Socialcomponents of Smart Systems. Question for the acceptabilityof such systems is how they can be designed so as to bedeemed trustworthy by users and social components within

the system. As we have discussed, trust is affected by thenormative expectations that a user has about a system and bythe process of belief formation about the performance of thesystem. The proposed formal model of trust is an importantstep in operationalizing the notion of trust, and to quantify andreason about trust/trustworthiness in complex systems.

To design more trustworthy systems, we will pursue partic-ipatory approaches that provide opportunities for the socialactors to dynamically contribute to defining the normativeexpectations of the system. We will also explore design mech-anisms to make these systems more transparent and enhancethe collective intelligence of social actors, so that they mayform more accurate beliefs about the future performance ofthe system, thereby reducing their vulnerability. We will alsoexplore system of systems design features that maximize theagency of social components in CPS3.

To this end, we will combine the formalization of trust withSystems of Systems Engineering methodologies [50], [51],[52], agent-based simulations of human behavior [53], [54],[55], and risk management [56], [57], for more trustworthyCyber-Physical-Social Smart Systems design, operations, andmanagement.

REFERENCES

[1] SRI, “Disruptive civil technologies,” 2008.[2] United Nations, “World population ageing 2013,” 2013.[3] T. Grandison and M. Sloman, “A Survey of Trust in Internet Appli-

cations,” IEEE Communications Surveys and Tutorials, vol. 3, no. 4,August 2000.

[4] C.-N. Ziegler and G. Lausen, “Propagation Models for Trust and Distrustin Social Networks,” Information Systems Frontiers, vol. 7, no. 4-5, pp.337–358, 2005.

[5] D. Artz and Y. Gil, “A survey of trust in computer science and thesemantic web,” Journal of Web Semantics, vol. 5, pp. 58–71, 2007.

[6] K. Govindan and P. Mohapatra, “Trust computations and trust dynamicsin mobile adhoc networks: A survey,” Communications Surveys Tutori-als, IEEE, vol. 14, no. 2, pp. 279–298, 2012.

5

[7] M. Burrows, M. Abadi, and R. Needham, “A Logic of Authentication,”ACM Transactions on Computer Systems, vol. 8, no. 1, pp. 18–36, 1990.

[8] U. M. Maurer, “Modelling a Public-Key Infrastructure,” in ESORICS’96: Proceedings of the 4th European Symposium on Research inComputer Security. London, UK: Springer-Verlag, 1996, pp. 325–350.

[9] P. Zimmermann, The Official PGP User’s Guide. MIT Press, 1995.[10] S. P. Marsh, Formalising Trust as a Computational Concept. Ph.D.

Thesis, University of Stirling, 1994.[11] J. Kleinberg, “Authoritative Sources in a Hyperlinked Environment,” in

Proc. 9th ACM-SIAM Symposium on Discrete Algorithms, 1998.[12] L. Page, S. Brin, R. Motwani, and T. Winograd, “The PageRank Citation

Ranking: Bringing Order to the Web,” Technical report, Stanford DigitalLibrary Technologies Project, 1998.

[13] S. D. Kamvar, M. T. Schlosser, and H. Garcia-Molina, “The EigenTrustAlgorithm for Reputation Management in P2P Networks,” in WWW ’03:Proceedings of the 12th International Conference on World Wide Web.New York, NY, USA: ACM, 2003, pp. 640–651.

[14] L. Mui and A. Halberstadt, “A Computational Model of Trust andReputation,” in Proc. 35th Hawaii Int. Conf. on System Sciences, 2002.

[15] M. Blaze, J. Feigenbaum, and J. Lacy, “Decentralized Trust Manage-ment,” in Proc. IEEE Symposium on Security and Privacy, 1996.

[16] M. Winslett, T. Yu, K. E. Seamons, A. Hess, J. Jacobson, R. Jarvis,B. Smith, and L. Yu, “Negotiating trust on the web,” IEEE InternetComputing, vol. 6, no. 6, pp. 30–37, 2002.

[17] P. Resnick, R. Zeckhauser, E. Friedman, and K. Kuwabara, “ReputationSystems,” Communications of the ACM, vol. 43, no. 12, pp. 45–48, 2000.

[18] P. Resnick and R. Zeckhauser, “Trust Among Strangers in InternetTransactions: Empirical Analysis of eBay’s Reputation System,” TheEconomics of the Internet and E-Commerce. Michael R. Baye, editor.Advances in Applied Microeconomics, vol. 11, pp. 127–157, 2002.

[19] C. Dellarocas, “The digitization of word of mouth: Promise and chal-lenges of online feedback mechanisms,” Management science, vol. 49,no. 10, pp. 1407–1424, 2003.

[20] J. Sabater and C. Sierra, “Review on computational trust and reputationmodels,” Artif. Intell. Rev., vol. 24, no. 1, pp. 33–60, 2005.

[21] A. Abdul-Rahman and S. Hailes, “A Distributed Trust Model,” in NSPW’97: Proceedings of the 1997 Workshop on New Security Paradigms.New York, NY, USA: ACM, 1997, pp. 48–60.

[22] B. Yu and M. Singh, “A Social Mechanism of Reputation Managementin Electronic Communities,” in Proc. of 4th Int. Workshop on Cooper-ative Information Agents, 2000, pp. 154–165.

[23] J. Golbeck, B. Parsia, and J. Hendler, “Trust Networks on the SemanticWeb,” in Proceedings of Cooperative Information Agents. Helsinki,Finland. Springer, 2003.

[24] A. Josang, S. Marsh, and S. Pope, “Exploring Different Types of TrustPropagation,” in Proceedings of the 4th International Conference onTrust Management (iTrust’06), 2006.

[25] J. Huang and D. Nicol, “A Calculus of Trust and Its Application to PKIand Identity Management,” in Proceedings of IDTrust’09, ACM DigitalLibrary, http://portal.acm.org/citation.cfm?id=1527017.1527021, 2009.

[26] L. Xiong and L. Liu, “PeerTrust: Supporting Reputation-Based Trust forPeer-to-Peer Electronic Communities,” IEEE Transactions on Knowl-edge and Data Engineering, vol. 16, no. 7, pp. 843–857, 2004.

[27] S. Park, L. Liu, C. Pu, M. Srivatsa, and J. Zhang, “Resilient trustmanagement for web service integration,” in Proc. IEEE Int. Conferenceon Web Services,, 2005.

[28] Z. Malik and A. Bouguettaya, “Reputation bootstrapping for trustestablishment among web services,” IEEE Internet Computing, vol. 13,no. 1, pp. 40–47, 2009.

[29] J. Huang and D. M. Nicol, “Trust mechanisms for cloud computing,”Journal of Cloud Computing, vol. 2, no. 1, 2013. [Online]. Available:http://www.journalofcloudcomputing.com/content/2/1/9

[30] Z. Yan, P. Zhang, and A. V. Vasilakos, “A survey on trust managementfor internet of things,” Journal of network and computer applications,vol. 42, pp. 120–134, 2014.

[31] S. Sicari, A. Rizzardi, L. A. Grieco, and A. Coen-Porisini, “Security,privacy and trust in internet of things: The road ahead,” ComputerNetworks, vol. 76, pp. 146–164, 2015.

[32] I.-R. Chen, F. Bao, and J. Guo, “Trust-based service management forsocial internet of things systems,” 2015.

[33] J. Glowacka, J. Krygier, and M. Amanowicz, “A trust-based situationawareness system for military applications of the internet of things,”in Internet of Things (WF-IoT), 2015 IEEE 2nd World Forum on, Dec2015, pp. 490–495.

[34] M. Deutsch, “Cooperation and Trust: Some Theoretical Notes,” inNebraska Symposium on Motivation, M. Jones, Ed., vol. X. UniversityNebraska Press, 1962, pp. 275–318.

[35] J. Rotter, “A New Scale for the Measurement of Interpersonal Trust,”J. Personality, vol. 35, pp. 651–665, 1967.

[36] N. Luhmann, Trust and Power. John Wiley & Sons Ltd, 1979.[37] D. Gambetta, “Can We Trust Trust?” in Trust : Making and Breaking

Cooperative Relations, D. Gambetta, Ed. Blackwell, 1988, pp. 213–237.

[38] R. Mayer, J. Davis, and F. Schoorman, “An Integrative Model of Orga-nizational Trust: Past, Present, and Future,” Academic of ManagementReview, vol. 20, no. 3, pp. 709–734, 1995.

[39] L. Zucker, “Production of Trust: Institutional Sources of EconomicStructure, 1840-1920,” Research in Organizational Behavior, vol. 8, pp.53–111, 1986.

[40] J. Lewis and A. Weigert, “Trust as a Social Reality,” Social Forces,vol. 63, no. 4, pp. 967–985, 1985.

[41] K. Blomqvist, “The Many Faces of Trust,” Scandinavian Journal ofManagement, vol. 13, no. 3, pp. 271–286, 1997.

[42] J. Huang and M. S. Fox, “An Ontology of Trust – Formal Semanticsand Transitivity,” in Proceedings of the 8th International Conference onElectronic Commerce. ACM, 2006, pp. 259–270.

[43] J. Huang, Knowledge Provenance: An Approach to Modeling andMaintaining The Evolution and Validity of Knowledge. Ph.D. Thesis,University of Toronto, http://hdl.handle.net/1807/11112, Dec 2007.

[44] J. Huang and D. Nicol, “A formal-semantics-based calculus of trust,”IEEE Internet Computing, vol. 14, no. 5, pp. 38–46, Sep. 2010.[Online]. Available: http://dx.doi.org/10.1109/MIC.2010.83

[45] A. Josang, “A logic for uncertain probabilities,” International Journalof Uncertainty, Fuzziness, and Knowledge-Based Systems, vol. 9, no. 3,pp. 279–311, 2001.

[46] A. Hajek, “Probability, Logic, and Probability Logic,” in PhilosophicalLogic, L. Goble, Ed. Blackwell Publishing, 2001.

[47] J. Y. Halpern and Y. Moses, “A guide to completeness and complexityfor modal logics of knowledge and belief,” Artif. Intell., vol. 54, no. 3,pp. 319–379, 1992.

[48] F. B. Schneider, Trust in cyberspace. National Academies Press, 1999.[49] A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr, “Basic concepts

and taxonomy of dependable and secure computing,” Dependable andSecure Computing, IEEE Transactions on, vol. 1, no. 1, pp. 11–33, 2004.

[50] C. Keating, R. Rogers, R. Unal, D. Dryer, A. Sousa-Poza, R. Safford,W. Peterson, and G. Rabadi, “System of systems engineering,” Engi-neering Management Journal, vol. 15, no. 3, pp. 36–45, 2003.

[51] A. V. Gheorghe and D. Vamanu, “Mining intelligence data in the benefitof critical infrastructures security: vulnerability modelling, simulationand assessment, system of systems engineering,” International Journalof System of Systems Engineering, vol. 1, no. 1-2, pp. 189–221, 2008.

[52] C. B. Keating and J. M. Bradley, “Complex system governance referencemodel,” International Journal of System of Systems Engineering, vol. 6,no. 1-2, pp. 33–52, 2015.

[53] D. Cetinkaya, A. Verbraeck, and M. D. Seck, “Model continuity indiscrete event simulation: A framework for model-driven developmentof simulation models,” ACM Trans. Model. Comput. Simul., vol. 25,no. 3, pp. 17:1–17:24, Apr. 2015. [Online]. Available: http://doi.acm.org/10.1145/2699714

[54] M. Seck, N. Giambiasi, C. Frydman, and L. Baati, “Devs for humanbehavior modeling in cgfs,” The Journal of Defense Modeling andSimulation: Applications, Methodology, Technology, vol. 4, no. 3, pp.196–228, 2007.

[55] G. Kolfschoten, S. Lukosch, and M. Seck, “Modeling collaborationprocesses to understand and predict group performance,” in Proceedingsof the 1st International Workshop on Semantic Models for AdaptiveInteractive Systems, ser. SEMAIS ’10. New York, NY, USA: ACM,2010, pp. 1–5. [Online]. Available: http://doi.acm.org/10.1145/2002375.2002376

[56] A. Gheorghe and R. Mock, Risk Engineering: Bridging Risk Analysiswith Stakeholders Values. Springer Science & Business Media, 2012,vol. 6.

[57] X.-B. Hu, A. V. Gheorghe, M. S. Leeson, S. Leng, J. Bourgeois, andX. Qu, “Risk and safety of complex network systems,” MathematicalProblems in Engineering, vol. 2016, 2016.

6