Collapse and reorganization patterns of social … and reorganization patterns of social knowledge representation in evolving semantic networks Kostas Alexandridisa, , Yiheyis Marub

Information Sciences 200 (2012) 1–21

Contents lists available at SciVerse ScienceDirect

Information Sciences

journal homepage: www.elsevier .com/locate / ins

Collapse and reorganization patterns of social knowledge representationin evolving semantic networks

Kostas Alexandridis a,⇑, Yiheyis Maru b

a Center for Marine and Environmental Studies (CMES), College of Science and Mathematics, University of the Virgin Islands, USAb CSIRO Ecosystem Sciences, Alice Springs, Australia

a r t i c l e i n f o a b s t r a c t

Article history:Received 7 April 2009Received in revised form 16 November 2011Accepted 26 February 2012Available online 14 March 2012

Keywords:Self-organizationSemantic network analysisNatural language processingCollective knowledge representationNetwork evolutionAgent-based modeling

0020-0255/$ - see front matter � 2012 Elsevier Inchttp://dx.doi.org/10.1016/j.ins.2012.02.053

⇑ Corresponding author. Tel.: +1 340 693 1376; faE-mail address: [email protected] (K. Alexandridi

This study introduces semantic network analysis of natural language processing in collec-tive social settings. It utilizes the spreading-activation theory of human long-term memo-ries from social psychology to extract information and graph-theoretic linguisticapproximations supporting rational propositional inference and formalisms. Using anempirical case study we demonstrate the process of extracting linguistic concepts from dataand training a Hopfield artificial neural network for semantic network classification. Wefurther develop an agent-based computational model of network evolution in order to studythe processes and patterns of collective semantic knowledge representation, introducingincidents of collapses in central network structures. Large ensembles of simulation replica-tion experiments are conducted and the resulted networks are analyzed using a variety ofestimation techniques. We show how collective social structure emerges from simple inter-actions among semantic categories. Our findings provide evidence of the significance of col-lapse and reorganization effects in the structure of collective social knowledge; thestatistical importance of the within-factor interactions in network evolution, and; stochas-tic exploration of whole parameter spaces in large ensembles of simulation runs can revealimportant self-organizing aspects of the system’s behavior. The last session discusses theresults and revisits the issues of generative semantic inference and the semantic networksas inferential formalisms in guiding self-organizing systemic complexity.

� 2012 Elsevier Inc. All rights reserved.

1. Introduction and rationale

1.1. Semantic networks and collective knowledge representation

Semantic networks present coherent and cohesive structures for knowledge representation (KR) and qualitative datamining tasks. Although often encountered as complex ontological hierarchies following strict rules of object inheritance[21,101], their most efficient and information-theoretic symbolism is their graph-theoretic representation as patterns ofinterconnected nodes (vertexes) and links (arcs). In such symbolic form, semantic networks encode a wide variety of seman-tic relationships, including linguistic approximations of knowledge and different types and scales of connectivity over com-plex hierarchical structures of subsumptional entities [20,62].

Semantic expressions can introduce knowledge approximations (also known as isomorphisms) to logic-based, rationalpropositional inference or formalisms [39,75]. According to Quillian [72], semantic networks based on linguistic assertionsform an abstraction theory of the structure of the human long-term memory, able to embody knowledge as a computational

. All rights reserved.

x: +1 340 693 1385.s).

http://dx.doi.org/10.1016/j.ins.2012.02.053

mailto:[email protected]

http://dx.doi.org/10.1016/j.ins.2012.02.053

http://www.sciencedirect.com/science/journal/00200255

http://www.elsevier.com/locate/ins

2 K. Alexandridis, Y. Maru / Information Sciences 200 (2012) 1–21

model of mind. The spreading-activation theory of semantic processing [8,9,26] studies the effects of priming in semanticmemory via associative semantic networks of interconnected relationships. These associations are stored and recalled inmemory altogether (retention–activation process). Semantic networks of memory processing have recently contributed toa new approach to learning theory, namely the connectivism as a learning theory [82]. Connectivism focuses on the under-standing of how informal learning patterns and associations (as complex adaptive systems) help people utilize and connectlearning and knowledge rather than information itself. In their graph-theoretic properties, semantic networks are proven tofollow scale-free distributions [12,73], and found to be strongly related to weak social ties in social networks of interactions[43,59].

In this paper, we will provide an evidence-based framework that goes beyond the construction of semantic networksfrom natural language knowledge representation [38,99], attempting to explore their evolution and their patterns of resil-ience and reorganization over a series of simulation-based experimentation techniques. A brief overview of the participatoryand bottom-up empirical construction of semantic networks for a case study will be provided, and the use of Hopfield-typeArtificial Neural Networks for training and extracting spreading-activation thresholds will be explored. Hopfield ANNs pres-ent a type of recurrent NNs with simple neurons and no hidden layers [17,52]. They utilize dynamic computation of auto-associative semantic rules in connecting synaptic nodes in the network and symbolize content-addressable memory systemswith binary threshold units. The iterative nature of semantic memory processing [48,92] and retrieval in Hopfield SemanticNets allows for finding a lower ‘‘energy’’ level for the network that self-organizes spreading-activation network patterns overknowledge hierarchies. The paper will showcase how semantic processing in semantic memory priming and retention, mod-eled using Hopfield-type ANNs allows for efficient and correct knowledge approximation and ontological representation ofdynamics in human cognitive processing [see, for example, 4,11]. It will also demonstrate how human memory self-organi-zation of semantic patterns improves informal learning and decision-making.

In terms of their social science context, the semantic representation discussed in this paper elicits collective social under-standing of contextual relationships. In other words, the emergence of collective social structure as a function of how closelyrelated are the mental models of individuals as members of the community or the social system under study. In this way, ourapproach to semantic network construction deviates from traditional cognitive approaches to knowledge representation.The strength of the semantic associations measured in our study depends on the degree to which such associations arewidely shared by other individuals in the community. The more semantic concepts are shared by more members, the morethey function at the collective social level of inference [38,81].

Sawyer [78] argues that traditional theories of emergence and the structural patterns from widely accepted system the-ories of complexity such as the chaos theory and the away-from-equilibrium dissipative structures of Prigogine [68,69] arehard to fit into the sociological and sociocultural realities of social emergence. He argues that the internal interactions amongthe components of the social system itself (e.g., individuals, institutions, social collective structures and norms) are far moresignificant from their perspective ‘‘sociobiological inflows’’ [78]. He also raises the importance of computational social sci-ence techniques on the study of macro-sociological phenomena and social mechanisms as a form of ‘‘social explanation’’[77]. A growing number of articles in the relevant literature illuminate the high added value of adopting and exploring com-putational methods in social, cognitive and psychological sciences [23,35,76,86]. Computational social science methods,techniques and experimentation allow the study of complexity, adaptation and emergence in social systems ranging fromthe individual to the collective level of inference. Parallel advances in computational algorithms and artificial intelligencemethodologies have provided an ever-expanding toolbox for computational social scientists in their attempts to studyand explain highly complex patterns of social interactions (e.g., [61,71,80,94,97,100]). The volatile and subjective natureof human judgments, attitudes, dispositions and behaviors, coupled with the deep uncertainties and incomplete informationcharacterizing the flows of real-world phenomena and their social and natural environments brings forth often implausiblechallenges for computational social scientists. It is not simply enough to describe ‘‘how’’ complexity in social systemic inter-actions emerges, but requires theoretical and evidence-based empirical arguments attempting to address ‘‘why’’ these com-plexity patterns emerge from social interactions, as well as for which scale prevalence of social inference (i.e., cognitive,individual, collective). Exploring the micro-to-macro links has been the ‘‘holy grail’’ of social science for the last few decades[32,35,44].

Patterns of collapse and reorganization in social systems can emerge and persist in parallel in a society at different scalesof social inference. For example, at the same time income inequality between the top 20% of the richest and the bottom 20%of the poorest in the US population has increased noticeably both in absolute numbers and in its global rankings, economicgrowth at the collective level and investment levels in monetary terms has risen considerably [89,93]. Under the social sci-ence context, from the perspective of the increasing number of individuals falling under the poverty line in US their individ-ual social experience could be characterized as a social collapse. On the other hand, from the perspective of the top globalcompanies as collective organizations, their collective social experience is characterized as reorganization. Both of these so-cial realities (individual and collective organizational) coexist in parallel social realities and form a part of a social holon,forming complex holarchies, the latter terms borrowed those terms from Lovelock’s and Margulis’s Gaia hypothesis [60].

1.2. Related work and structure of the study

The empirical research findings from studying collective social processes, including collective knowledge representationflows, suggest that the transition from the individual to collective structure emerges alongside with the emergence of

K. Alexandridis, Y. Maru / Information Sciences 200 (2012) 1–21 3

power-law type degree distributions in social semantic networks. Our recent study of collective semantic associations in aremote desert Australian Outback community (Anmatjere region of central-north Australia) measured and analyzed a num-ber of collective social aspects in relation to sustainability of employment-based livelihood outcomes and opportunitiesusing semantic network analysis. Our results [5], show that collective social aspects of livelihoods such as social cohesion,centrality and social brokerage roles, along with their patterns of interactions across individuals and groups forming cohe-sive subgroups can be studied extensively using semantic network analysis. The Anmatjere community was a part of a set ofremote Australian communities subjected to wider policy government interventions [87] aiming on reorganizing andenhancing collective social structures, roles and responsibilities. What became widely known in Australia as the NorthernTerritory Intervention, consisted of policy measures that removed central community and local governance institutions, of-ten met with wider social and community distrust and a very high level of uncertainty about the future. In the midst of thesechanges our field work interviews and focus group meetings was conducted, but did not aimed to study the near- and longer-term effects of such changes in the collective social understanding of livelihoods in the region. The research effort we reportin this paper utilizes the empirical findings of the Anmatjere semantic network analysis and introduces a series of additionalquestions that attempt to widen our theoretical and policy-related understanding of social change in relation to collectivelivelihoods.

We thus introduce a computational simulation framework that conducts social experiments over semantic networksstructured in ways to resemble the social structure encountered in our empirical social settings. Within such an experimen-tal computational social framework we conceive each collective semantic entity (i.e., network node) as an artificial agent,equipped with a number of key social metrics that signify its collective social importance, role and position in the overallsemantic social structure. We also introduce a number of externally imposed changes in the network such as removal ofkey central nodes and collapses with varying degrees of severity and/or frequency of incidental occurrence. Finally, we pro-vide the simulation agents (network nodes) with a response/feedback mechanism as the ability to restructure their associ-ation with other (similar) nodes following externally imposed events. Our aim is to study the following research questions:

(a) How semantic structure emerges from simple semantic interactions/associations (Section 2)?(b) What are the effects in collective semantic social structure following collapses of key or very central social institutions

or concepts (Section 3.1)?(c) What is the magnitude of these effects when we vary the frequency and the severity of these interventions (Sections

3.1 and 3.2)?(d) Given a set of simple feedback/response rules for the agents, how does social structure emerges and reorganizes if at

all (Section 3.3)?(e) What are conditions under which global (network-wide, that is, at the whole social system level) or local (cohesive

group-specific) reorganization patterns emerge and propagate through the evolving networks (Sections 3 and 4)?(f) What can we learn about guided-self organization in the context of knowledge representation in collective social sys-

tems (Section 4)?

The structure of the paper is as follows: Section 2 presents the basic construction of the semantic network inference, andthe basic simulation framework for testing the emergence of self-organization. Specifically in Section 2.1 we present the ba-sic inferential and algorithmic summary of the semantic network construction from natural language textual information,and its calibration for extracting the main network nodes (semantic ontologies) and the network links (semantic weightsor strengths). Section 2.2 presents a computational social simulation ensemble using an agent-based model for testingthe effect of different factors affecting the emergence of self-organizing patterns in similar semantic networks. Section 3 pre-sents the basic analysis of the simulation ensemble results. The hierarchical decomposition of network trees, the emergenceof power law distributions in semantic networks and the effects of different factorizations of network evolution (collapseintensity and frequency, and reorganization potential) are provided in Section 3.1. A longitudinal panel model analysis is pre-sented in Section 3.2 testing and estimating the combined effects and combination of interactions among the network sim-ulation ensemble factorizations, allowing a robust statistical framework that supports the proposition of emergentproperties in complex systemic interactions in simulation ensembles. Section 3.3 goes further and test the effects of the para-metric complexity in the emerging evolution of semantic network simulation ensembles using a GLM univariate estimatormodel. The final analysis Section 3.4 uses a multilayer preceptor ANN to group and estimate the accuracy of the factorizationin the simulation ensembles network. The final Section 4 of the paper presents and summarizes the important findings of theanalysis (Section 4.1) and provides a series of inference in the study of guided self-organization in semantic systems of col-lective knowledge representation (Section 4.2). The outline of the paper is shown in the following Fig. 1.

2. Simulating semantic network emergence

2.1. Extracting semantic networks from natural language texts in the Anmatjere case study

A three-stage process for extracting and analysing semantic knowledge representation flows has been employed. First, anumber of interviews were conducted during field visits in the region with community members (n = 72). Two additional

Fig. 1. An outline of the organization and the inferential framework of the current paper.


focus group meetings followed the interviews (for men and women, given social conventions in the Aboriginal culture prom-inent in the region) that provided the opportunity for discussions and social deliberations among community members. Thetranscribed texts of the interviews and focus group meetings along with written summaries of discussion points have beenused to extract an initial set of concepts present in the natural language texts using frequency-based linguistic data miningtechniques (see for example [10,24,31]). The initial set of extracted concepts and their frequency weights was used as train-ing inputs in the implementation of a Hopfield ANN algorithm for grouping those extractions to semantic categories. TheHopfield model produced a set of estimated semantic weights, by calibrating the spreading-activation threshold for inclusionin the network, and then optimizing the weights given that threshold. The use of the Hopfield-type ANN models is shown toproduce good estimations in use with the spreading-activation theory of semantic processing [42,98]. The algorithm uses arecursive estimation for optimizing the objective function:

1 The

E ¼ �12Pki<j

aij � si � sj þPk

iwt � si ð1Þ

where si,j are the initial frequencies of each node n in the text; wt is the spreading-activation threshold, and aij is the activa-tion weight function between nodes ni, nj and is calibrated such that:

aij ¼1 if wij > wt

0 if wij < wt

wij if wij ¼ wt

8><>: ð2Þ

where wij: semantic weight between nodes ni, nj.The objective function shown in Eq. (1) is also known as the Lyapunov equation whose asymptotic attractor properties as

widely known and reported [36,42,96]. The auto-associative energy function estimation is capable of global optimization asit is shown to have properties resembling universal Turing computation [19,34]. The final output of the Hopfield ANN esti-mator is the calibrated semantic activation weights that optimize the objective function in Eq. (1). Further details of the ma-chine learning categorization properties can be found in the relevant literature of semantic networks [6,27,88].

As can be seen from the properties of the Hopfield ANN estimator, the spreading-activation threshold value, wt is exog-enous to the model estimation, that is, has to be determined a priori. In the Anmatjere semantic data, we imputed the algo-rithm for a vector of thresholds, ranging from 2 to 20.1 The results in terms of the estimated number of semantic categories(concept groups) and the estimated mean number of ties (links among categories), is shown in the top part of Fig. 1. Both trendsshow the prevalence of power-law distributions in the number of estimated semantic network sizes. We can think of thespreading-activation threshold as an indicator of ‘‘collectiveness’’ in social and community knowledge representation observedfrom our data. Increasing levels of threshold capture more widely held attitudes, beliefs or norms shared by wider groups ofparticipants in the study. The presence of these power-law distributions shows how only a very small number of semantic cat-egories and their knowledge-flows have wider influences on the broader collective social system.

Computing the ratio between extracted semantic categories (nodes) and semantic ties in our data showing in the bottompart of Fig. 1 allows us to observe the semantic emergence of a tipping point across spreading-activation thresholds. As wemove from the individual to the collective level of semantic inference (increased threshold levels), the density of ties in-creases initially. This means that shared knowledge flows expand among widening cohesive subgroups of community mem-bers, until a tipping-point is reached (where wt = 8) above which knowledge flows spread by few key collective semanticcategories. Evidence from social network theory suggests that this might be the point where small-world network dynamicsemerge [16,91], and the power of weak social ties exceeds collective social structure and institutions [18,46].

An example of the emerging Anmatjere semantic network for a threshold of wt = 5 is shown in the following Fig. 2. Nodesizes are varied based on the computed Freeman’s degree centrality coefficient [90], and show how nodes with high degree

value of wt = 1 is trivial, as it produces all possible extracted concepts from data, with no classification groupings.

2

4

6

8

10

Sem

antic

Tie

Den

sity

2 4 6 8 10 12 14 16 18 20

Spreading-Activation Semantic Thresholdbandwidth = .8

Mean adjusted smoothLowess smoother

Fig. 2. Top left and right: Distribution of the observed number of nodes and ties across spreading-activation thresholds in the Anmatjere semantic networkrespectively; bottom: Distribution of the observed tie density across increasing spreading-activation threshold levels in the Anmatjere semantic networkdata.


centrality represent higher levels of collective social concepts such as the Anmatjere community, the (Aboriginal cultural)ways of doing things, the concept of livelihood activities, and the Anmatjere Shire Council.

Further analysis on the structure and social properties of the emerging Anmatjere semantic network can be found in thepaper by Alexandridis et al. [5], and exceeds the scope of this paper. In the next sessions, we will attempt to go beyond theempirical snapshot of social structure in the Anmatjere region and perform additional analyses that will explore more dy-namic dimensions of semantic emergence and evolution.

2.2. From semantic extraction to computational social simulations

The emergence of computational social science techniques, especially in the domain of social simulations has gained sig-nificant recognition in the last decade. Social simulations using artificial agents have been used extensively to aid our com-prehension of both individual and social dynamics, as well as to unravel the complexity and patterns of interactionsemerging at different social settings and situations [7,33,49,57]. When social simulation approaches are combined with at-tempts to achieve computational robustness especially in the face of policy-related research questions, increases the value ofthe scientific inference. Lempert, Popper and Bankes from Rand corporation [58] introduce the concept of robustness in sim-ulation ensembles as powerful tools to perform sweeps of parameter spaces for desirable conditions or combination of con-ditions. They provide four basic principles or rules of thumb for achieving a robust framework of computational simulationensembles: (a) exploring large parameter spaces by considering ensembles of very large simulation runs; (b) striving forrobustness rather than optimality in computational strategies, i.e., conditions under which solutions persist rather thanbeing optimal; (c) utilizing adaptive strategies seeking robustness, and (d) performing backwards-inference, e.g., askingwhat conditions fail to meet assumptions set forth by the simulations. In this analysis, we employed these rules to developour simulation framework for the semantic emergence.

We introduced an ensemble of agent-based simulations using the NetLogo modeling environment [63]. Each artificialagent in the model represents a semantic node. At each simulation step a new agent is emerging and trough an iterative algo-rithm links with another node. The linking algorithm was designed in a way to replicate the power-law dynamics observedin our empirical Anmatjere semantic network study. Barabási has demonstrated how preferential attachment gives rise topower-law, scale-free distributions [13,14], thus we used a preferential attachment algorithm for performing this task.Specifically, at each time step a new agent links with a node sampled from a probability distribution proportional to the

accommodation

activity

admin

AileronAlice Springs

Alyuen

Anmatjere

carcattleCDEP

centrelink

child

childcare

client

clinic

community

coordinator

councilcountry

culture

daughter

day

drilling

drinking

education

elder

emergency

exploration

family

food

government

grape

health

income

individual

knowledge

life

machinery

man

money

month

Northern Territoryoffice

older

opportunityparentperson

Pmara Jutunta

region

road

role

skill

supervisor

teacher

Ti Tree

town

ways of doing things

whitefellas

wife

Wilora

woman

worker

year

Fig. 3. Graph-theoretic display of the Anmatjere semantic network trained from natural language textual data for spreading-activation threshold valueh = 5. Node sizes represent nodal degree centrality measures.


networks’ tie density. The specific probability distribution is chosen as a model parameter of interest. Agents have the abilityto measure and monitor key structural metrics of their status in their network at each step of the simulation, including theirnodal centrality, the number of nodes to which they are connected, as well their actor-level degree and betweenness cen-trality [29,90]. At this reference simulation level, the model runs by evolving the network size. In this instantiation of themodel, the network size is proportional to the time steps of the simulation.

In addition to the reference level, we introduced a set of parameters that alter the structure of the network. Specifically,the simulation framework allows for introducing the following additional variables:

(a) A collapse event, with frequency ct = t/T. For a simulation with total steps T, at given time step intervals t/T, a collapseincident occurs, such that nodes with degree centrality above a certain level are removed from the network. Thisdegree centrality level is determined with binary probability:

PrðcollapseijctÞ ¼1 if di > 1þ ð1� sÞ �maxðdnÞ0 otherwise

�ð3Þ

where di is the degree centrality of a node i; the term max(dn) denotes the maximum nodal degree across all actors in thesimulation, and s is the severity of the collapse incident. When the severity is low, only the central actor is removed, butas the severity increases, the removal threshold becomes lower, and more actors are removed from the simulation.

(b) A severity of collapse level, s 2 [0, 1]. The level determines the magnitude and effect of the collapse incident impacts inthe network, as seen before.

(c) An actor reorganization level r 2 [0, 1]. Following every collapse incident, the agents (actors) of the semantic networkstill remaining in the simulation are given the opportunity to re-establish lost links resulting from the collapse. Onlythose agents that have centrality above the mean network-level centrality are allowed to reconnect. During this stage,the agents pick randomly from one of their neighboring nodes and establish a connection. An implicit aspect of thesimulation (as a result of the spring-embedding network layout) is that neighboring nodes have Euclidean distancesinversely proportional to their centrality. Thus, this stage allows similarly central semantic concepts to connect.

These three external parameters (i.e., c, s, r) can be determined interactively from the user. When the model runs contin-uously at intervals t0 = t0 + ct, collapses are introduced and subsequent nodal reorganizations occur. A snapshot of the user’sinterface of the NetLogo model is shown in the following Fig. 3.


2.3. Generating simulation ensembles using repeated Monte Carlo replication runs

For the remaining analysis of this paper, we generated two major simulation ensembles of Monte Carlo experiments usedfor different purposes of the analysis.

� Ensemble No. 1: reference simulation runs (without collapse incidents). A 100 simulation steps � 100 replication runs.Thus N1 = 100 � 100 = 10,000 data points over 100 semantic networks.� Ensemble No. 2: analysis simulation runs (with collapse incidents). We ran experiments of 100 step simulation over 100

replication runs each and tested all possible combinations for:

c ¼ f10;40;70;100gs ¼ f0:1;0:25;0:5;0:75;1:0gr ¼ f0:1;0:25;0:5;0:75;1:0g

thus N2 = 100 � 100 � 4 � 5 � 5 = 1,000,000 data points over 10,000 semantic networks.For each of the data points in the two simulation ensembles, the simulation reported a total of 18 variables related to the

state of the actors in the simulation. In addition, at the end of each replication run (after 100 time steps) we exported the fullgraph network for further analysis. The simulation ensemble results are analyzed in the next session. An important consid-eration that restricted larger ensembles is the capability of spreadsheets and statistical packages to analyze larger than theones computed data samples. The second ensemble of simulation runs took approximately 8 h of simulation in an ordinaryIntel Core dual-CPU laptop running at 2.00 GHz frequency with 2 GB of RAM.

3. Analysis of simulation results

3.1. Simulation ensemble trend analysis

3.1.1. Semantic network evolution without collapsesAn example of the simulated semantic networks at different stages of network evolution is shown in Fig. 4. We used a

graph root-embedding algorithm [41,53] to display the hierarchical decompositions of semantic relationships in the net-work. The nodes and links of the semantic network are arranged from semantic parents to semantic children categories.The effect of power-law distribution over network degrees is apparent in the four reported stages of the network evolution.As the network grows, from 25 nodes to 100 nodes, we can see that the mean degree of separating the top nodes from themajority of the rest of the nodes does not change. Even at t = 100, most of the nodes appear to be 2–3 degrees from the most

Fig. 4. A screenshot of the NetLogo Simulation model used for the analysis of the semantic network patterns.


central one. We can also see how weak social ties emerge in the network, as subgroups of nodes form tightly connected clus-ters that are only weekly connected with more collective social clusters.

Another interesting property of the emergent semantic social structure comes from the spreading-activation theory oflong-term memory [26]. As spreading increases from the bottom to the top of the semantic topology, activation onlyfunctions in top-down flows (i.e., activating a node in the middle of the hierarchy also spreads to its children, but doesnot affect its parent nodes). In the social context of our simulation experiments, knowledge flows influence (spread) acrosssub-hierarchical social structures. Social structures such as the one emerging in Fig. 4 because of their scale-free propertiesare highly sensitive to structural changes at the top of the hierarchical decomposition.

In the first ensemble of simulation replication experiments we tested the effects of the semantic network evolution to keysemantic social network parameters. Plotting the results of the evolution of the simulation span we can reveal the dynamicsof temporal emergence in the semantic networks under study. Fig. 5 shows that a core set of semantic nodes are required forobserving scale-free distributions. Most of the simulated networks exhibit those properties when maximum nodal degreereaches a threshold between 8 and 12 degrees of centrality. If we recall the results from the spreading-activation thresholdcalibration of the Anmatjere semantic network in Fig. 1 (h = 8), we can see that our simulation findings provide a more robustconfirmation of the level of semantic activation in which collective social knowledge flows spread throughout thecommunity.

In order to provide more statistically robust evidence of the previous proposition, we performed a power-log regression ofthe mean observed network degree over the binned observed frequencies of network size from our simulation experiments.In each time step we computed the frequencies of nodes with various degrees, and averaged over the 10,000 simulated steps(for the exponential family of simulation runs). The regression results shown in Table 1, indicate a very good fit (adjustedR2 = .919), and the t-statistic for the effects of the independent variable to explain the variance in degree distribution is sta-tistically significant (p < .001).

Fig. 6 plots the predicted power-log regression along with the simulated mean network degrees of the dependentvariable.

3.1.2. Introducing collapses in semantic network evolutionThe introduction of collapses in semantic network structure has significant effects on the emergent structure of the evolv-

ing networks. Depending on different combinations of simulation parameters, those effects vary both in magnitude and

Fig. 5. Hierarchical decomposition of the evolution of a simulated semantic network over 100 simulation steps, using a root-embedding algorithm. Thestrength of collective social structure increases from children to parent nodes.

Table 1Power regression fit estimation statistics.

Unstandardized coefficients Standardized coefficients t Sig.

B Std. error Beta

ln(Network degree (Binned)) �3.06 0.226 �0.961 �13.533 .000(Constant) 207.292 99.211 2.089 .054

R = .961, R2 = .924, adj. R2 = .919. S.E. = .720.


spread across simulation time and semantic space. They range from network fragmentation to full systemic global networkcollapses. On the other hand, under certain conditions, global network-wide systemic reorganization emerges and propa-gates. Fig. 7 provides a series of visual examples of graph-theoretic layouts of network structures drawn from the resultsof the ensemble simulation runs. It would be impossible to graph the entire sweep of parameter space in this paper, sowe just provide an illustrative example of potential outcomes. Due to the multi-dimensional character of the simulationruns, all four example networks displayed in Fig. 7 have been sampled from results with the same moderate frequency ofcollapse c = 40, that is a collapse incidence occurs every 40 steps of the simulation run. Consequently the networks shownhere experienced two collapses over their total number of runs (at t = 40, and t = 80, respectively). The network growth issampled at four equal intervals not directly following collapse incidents. We employed a graph-theoretic spring electricalembedding algorithm with inferential distance of 2.0 using the Combinatorica package for Mathematica [67].

In the top row of the graph network matrix, a reference simulation run is provided (no collapses) for comparison reasons.When node reorganization levels are moderate and the severity of collapses is low (second row) despite minor fragmenta-tion of the semantic structure the network recovers and reorganizes into more cohesive social structure. Increasing theseverity of collapses in the network has dramatic effects on both the structure of the network and the ability to recover,as we experience system-wide fragmentation of collective structures and only limited and weakly connected cohesive sub-groups remain in the network. Further increasing the adaptive ability of the nodes to reorganize at moderate (and high) col-lapse severity levels does locally recover some centralized network structure, but those networks experience loss ofresilience and exhibit structural sensitivity in subsequent collapse incidents.

The primary results of the simulations in the second ensemble of simulations are summarized in the following Figs. 8 and 9,in which a more precise picture is provided for different combinations of parameters in the model ensemble runs.

The interplay of patterns in the simulated mean nodal degree between severity of collapse and reorganization levels isshown in Fig. 8. Four different regions of interest can be identified in the results:

(a) Local reorganizations (low severity, low reorganization potential): the system is robust, but not resilient (relativelysensitive) and bounces back from events with relative minor severity, but the majority of networks remain to the samestate.

(b) System-wide (global) reorganizations (low severity, high reorganization potential): the system undertakes state tran-sitions to more efficient network configurations. The relative minor severity levels trigger system-wide reorganiza-tions that exhibit high level of social resilience.

(c) System-wide (global) collapses (high severity, low reorganization potential): the system undertakes state transitionsto less efficient network configurations. The yielded networks either physically collapse or are extremely fragmented.The effect of severe events pushes the network in states that cannot recover from. The social system completely lackssocial resilience.

(d) Local collapses (high severity, high reorganization potential): the system shows a level of robustness that keeps it fromfull collapse. The majority of the networks maintain their overall state, but tend to be relative sensitive.

A similar set of interactions is observed for the simulated mean degree between severity and frequency of collapses in thesemantic networks (Fig. 9). In this instance, collapse incidents characterized by high frequency of occurrence and low

emergence of scale-free distributions

0

5

10

15

20

Max

imum

nod

al d

egre

e

0 20 40 60 80 100

Simulation step

Fig. 6. Emergence of scale-free distributions over the growth of semantic networks.

Fig. 7. Estimated power distribution of nodal degree in the simulation ensemble.


severity show higher mean degree centrality than the ones with moderate or low frequency overall. On the other hand, veryfrequent collapse incidents with high severity levels lead to systemic collapses of semantic social structure, and very unsta-ble dynamics when the frequency is moderate or low.

3.2. Modeling longitudinal effects on panel network variables

One of the important research questions in this analysis is to understand the effects of the severity of collapse and theconsequent reorganization patterns on the structure of the collective social structure in the semantic networks under anal-ysis. To the extent that such semantic networks encapsulate the strength and pattern of collective and joint social knowledgerepresentation within a community, the maximum degree of the vertexes in the network captures the power of the mostjoint held semantic concept. At the Anmatjere semantic network case study examples of central concepts with high degreecentrality are the sense of community, the Aboriginal ways of doing things, the Aboriginal Shire Council, and the institutionof family at the collective social level. Thus testing the effects in the overall network evolution when one or more of thesecentral concepts are removed and the collective knowledge representation is called to reorganize allows to demonstrate howrobust or resilient collective social structures are in such changes.

A useful approach for measuring the quantitative strength of those effects is to perform a panel regression of the maxi-mum degree centrality as a dependent variable over the collapse and reorganization levels as independent variables. Giventhe structure of the second ensemble of the simulation experiments (100 simulation steps � five collapse levels � five reor-ganization levels � four collapse frequencies � 100 replication runs = 1,000,000 networks), a longitudinal panel analysisseems to be an appropriate method of analysis. We defined the simulation steps as out time variable and the replication runsas the longitudinal panel variable and we estimated a GEE population-averaged model regression using the frequency of col-lapse as the exposure (incidence) variable. GEE PA models have been shown to produce valid estimates of the effect of theindependent variables to the dependent variable at the sampled population level under study [2,51]. The general equationfor the GEE population-averaged model is

gfEðyitÞg ¼ xitb; y � FðhÞ ð4Þ

where

g{�} is the assumed link function, e.g., for the power (2) link function g{y} = y2;E(yit) is the mean value of the estimated parameter of interest across i = 1, . . . , m group (panel) identifiers and overt = 1, . . . , ni simulation steps;xit is the panel vector of m observations by ni simulation steps for the dependent variable to be estimated, and;F(h) is the assumed probability distribution family over which the variable of interest is distributed. For example, for thePoisson distribution F(h) � Poisson with h = Pr (X = exp (z)).

The model estimates via maximum-likelihood the conditional probability:

PrðYit ¼ yitjxitÞ ¼ Fðyit ;xitbþ v iÞ ð5Þ

The assumed correlation structure for the error variance term vi can take various forms. Two within-group error variancecorrelation matrix (Rt,s) forms are examined for this model, namely the exchangeable equal-correlation structure shown inEq. (6), and the AR (1) auto-regressive correlation structure, given in Eq. (7).

Fig. 8. A snapshot of the network evolution in the semantic network simulation ensemble. Each row displays the evolution of a single network at 25simulation step intervals for different parametric conditions of the severity of collapse and reorganization simulation properties. The first row of the graphmatrix is a reference simulation (no collapses).


Rt;s ¼1 if t ¼ s

q otherwise

�ð6Þ

Rt;s ¼1 if t ¼ sqjt�sj otherwise

�ð7Þ

We tested 13 alternative model configurations and selected the model structure with the best overall fit on the popula-tion of networks under study. All models were statistically significant but with varying strength measured by their estimatedWald chi-square statistic. The results are shown in the following Table 2, sorted by their overall estimation performance. Theidentity link represents linear regression models, the log link represents logarithmic estimations, and the power link a powerregression with exponent 2. We also tested models from three family of distributions, namely the Normal (Gaussian), thePoisson, and the Gamma distributions. We also tested different correlation structures for the variance error term of the panelregression, the simple (exchangeable) structure, and the auto-regressive terms (of orders 1 and 2).

The model fit comparisons indicate that the best fit for the model is an AR (1) autoregressive power (2) Poisson panelregression model. The best model’s estimated Wald chi-square statistic yields 18.8% better score than the second-best

11.522.53

11.522.53

11.522.53

11.522.53

11.522.53

0 50 100 0 50 100 0 50 100 0 50 100 0 50 100

0.1, 0.1 0.1, 0.25 0.1, 0.5 0.1, 0.75 0.1, 1

0.25, 0.1 0.25, 0.25 0.25, 0.5 0.25, 0.75 0.25, 1

0.5, 0.1 0.5, 0.25 0.5, 0.5 0.5, 0.75 0.5, 1

0.75, 0.1 0.75, 0.25 0.75, 0.5 0.75, 0.75 0.75, 1

1, 0.1 1, 0.25 1, 0.5 1, 0.75 1, 1

(a) (b)

(c) (d)

Pred

icte

d m

ean

noda

l deg

ree

Seve

rity

leve

lReorganization level

Simulation step

Graphs by Severity level and Reorganization levelLines represent fitted values over 100 x 100 replication runs

Fig. 9. Evolution of mean degree as a function of the severity of collapse and reorganization levels in the simulated semantic networks.


model. The full estimation results for the model with the best fit are shown in the following Table 3. Both model parameters,namely the reorganization level and the severity of collapse level show statistical significant effects on the maximum nodaldegree of the network, since Pr(z > v2) < 0.001. Furthermore, the strength and direction of the computed Z statistic showsthe negative effect of the severity level and the positive effect of the reorganization level to the achieved max degree of thenetwork in our data.

Overall the analysis in this session demonstrated that (a) the nature of the relationship between the observed distributionof the maximum network degree as a function of the reorganization and severity of collapses can best be modeled using apower-law of exponent 2. This result reaffirms theoretical arguments related to network degree evolution [13,55]; (b) Bothreorganization and collapse levels have statistically significant opposite effects to the observed network degree, and (c)Those statistically significant effects follow are auto-correlated, that is, dependent on the temporal evolution of the network.

Table 2Model estimation results for the GEE population-averaged panel regression.

Model Link Family Correlation structure Wald v2 Relative change (%) Prob > v2

11 Power (2) Poisson AR (1) 3131.54 18.8 0.00003 Power (2) Gaussian Exchangeable 2637.02 3.3 0.0000

13 Power (2) Poisson AR (2) 2553.19 0.9 0.000010 Power (2) Gaussian AR (1) 2529.91 0.1 0.0000

6 Power (2) Poisson Exchangeable 2528.03 26.2 0.00009 Power (2) Gamma Exchangeable 2003.98 3.8 0.0000

12 Power (2) Gamma AR (1) 1931.13 24.5 0.00001 Identity Gaussian Exchangeable 1551.04 6.8 0.00004 Identity Poisson Exchangeable 1452.77 6.1 0.00005 Log Poisson Exchangeable 1369.86 94.2 0.00002 Log Gaussian Exchangeable 705.45 10.4 0.00007 Identity Gamma Exchangeable 638.82 3833.6 0.00008 Log Gamma Exchangeable 16.24 0.0 0.0003

Table 3Best model estimation results for the panel Poisson power auto-regressive model.

GEE population-averaged model Number of obs = 1,029,836Group and time vars: Repl #, Step # Number of groups = 10,000Link: Power (2) Obs per group: min = 99Family: Poisson avg = 103.0Correlation: AR (1) max = 106

Wald v2 = 3131.54Scale parameter: 1 Pr > v2 = 0.0000

Max Degree Coeff. Std. error Z Pr > z 95% Conf. interval

Reorganization level 2.3254 .100468 23.15 0.000 2.1285 2.5224Severity level �5.1485 .103321 �49.83 0.000 �5.3510 �4.9460_constant 37.8565 .488887 77.43 0.000 36.8983 38.8147Frequency of collapse (exposure)

Table 4Effects and significance of factor interactions in the GLM univariate model analysis. Tests of between-subjects effects.b

Source Type III SSE df MSE F Sig. Partial etasquared

Noncent.parameter

Observedpowera

Dependent variable: maximum nodal degree (Binned)Intercept Hypothesis 17754809.131 1 17754809.131 57.963 .000 .891 57.963 1.000

Error 2169748.775 7.083 306313.687Reorganization level Hypothesis 127470.050 4 31867.513 18.128 .000 .793 72.512 1.000

Error 33190.805 18.881 1757.924Severity level Hypothesis 866992.454 4 216748.113 19.497 .000 .826 77.989 1.000

Error 182785.601 16.442 11116.841Simulation step Hypothesis 281694.165 4 70423.541 5.624 .003 .524 22.497 .941

Error 255787.996 20.428 12521.471Reorganization level Hypothesis 3218.363 16 201.148 4.125 .000 .508 66.006 1.000⁄Severity level Error 3120.575 64.000 48.759Reorganization level Hypothesis 25689.256 16 1605.579 32.928 .000 .892 526.854 1.000⁄Simulation step Error 3120.627 64.000 48.760Severity level⁄ Hypothesis 175438.866 16 10964.929 224.872 .000 .983 3597.948 1.000Simulation step Error 3120.693 64.000 48.761Reorganization level Hypothesis 3120.712 64 48.761 31.542 .000 .002 2018.685 1.000⁄Severity level⁄

Simulation stepError 1534892.940 992,871 1.546

a Computed using alpha = .05.b Weighted least squares regression – weighted by frequency of collapse (incidence).


3.3. Predicting effects of parametric complexity to network evolution

In order to demonstrate the relative magnitude and complexity of interactions among the previous model parameters(independent variables) we conducted an additional GLM univariate analysis on the data ensemble. We tested the nullhypothesis about the effect magnitude of the independent variables (reorganization level, severity level and frequency ofcollapse) on the means of alternative groupings of the observed maximum degree. We de-trended the effects of the temporalevolution of the network by specifying the categorical binning of simulation steps (6 20, 21–40, 41–60, 61–80, 81–100) asfrequency weights for the univariate analysis of variance.

The Levene’s test of equality of error variances in the model, testing the null hypothesis that the observed error varianceof the maximum network degree is equal across groups produced a statistically significant result (F = 580.289, p < .0001). Thetest statistic allows us to reject the null hypothesis. The analysis of the effects and significance of between-factor interactionsin the univariate model is shown in the following Table 4. Whilst almost all of the 1, 2 and 3-way effects (with the exceptionof the simulation step) were found statistically significant, only the 2-way and 3-way factor interactions were producingestimates that minimized the mean squared error estimates. The estimated partial Eta-squared statistic g2

p

� �measures

the strength of the association among factors, as:

g2p ¼

SSeffect

SSeffect þ SSerrorð8Þ

The computer g2p results from Table 4 show that the two-way effects of increased severity or reorganization levels as the

network evolves (across increasing simulation steps) have very high strength of association (�.9 and above).The effects of factor interactions in the estimated maximum network degree are shown in the following Fig. 10. As sever-

ity level increases the marginal mean of the max degree decreases significantly, but as reorganization levels increase those

11.522.53

11.522.53

11.522.53

11.522.53

11.522.53

0 50 100 0 50 100 0 50 100 0 50 100

0.1, Low 0.1, Moderate 0.1, High 0.1, Very High




1, Low 1, Moderate 1, High 1, Very High

(a) (b)

(c) (d)

Pred

icte

d m

ean

noda

l deg

ree

Seve

rity

leve

lFrequency of collapse

Simulation step

Graphs by Severity level and Frequency of collapseLines represent fitted values over 100 x 100 replication runs

Fig. 10. Evolution of mean degree as a function of the severity and frequency of collapse in the simulated semantic networks.


effects are significantly contained, especially when reorganization levels exceed the .75 level. At those levels of reorganiza-tion, the network whilst losing their centrality strength (smaller maximum degrees), it does not experience regime shifts orgeneralized state transitions as the ones emerging at lower levels of reorganization when severity levels are high.

3.4. Using a Multilayer Perceptron (MLP) classification algorithm

In the final analysis stage of the simulated network ensemble we want to examine further the structure and pattern ofinteractions among the model run parameters (reorganization levels, severity levels, frequency of collapses, and temporalsimulation steps). So far, we showed that the effects of these parameters are significant predictors of the semantic structure(in the longitudinal panel analysis), and that cross-factor interaction effects influence the emerging variance of observed pat-terns in semantic social structure. One further critical question set forth in the introduction is yet to be answered: Whichcombination of factors enable or disable the emergence of system-wide self-organizing patterns across semantic social struc-ture? Attempting to provide an answer to this question is far from trivial task, as it involves understanding the specific pat-terns of interactions among the model parameters and how they affect the observed results. The previous analysis stepsshowed also that such interactions are nonlinear and nonmonotonic. Traditional statistical inference does not provide uswith tools to evaluate those patterns of interactions [3,22]. Machine learning algorithms and data mining techniques havebeen developed and showed able to overcome the problems of increased stochasticity and multidimensionality in databases[50,54]. We employed a pattern learning and estimation of the factorial semantic interactions by the use of an artificial neu-ral network Multilayer Perceptron (MLP) model. Multilayer perceptron models are pattern classifier algorithms capable tosolve blind-source separation type problems in an efficient and probabilistic manner [28,45].

We partitioned the data on two datasets, for training and testing purposes of the MLP classification model. Of the total992,996 valid (non-empty) data points for the dependent and independent variables, 70% of the data (694,752 cases) wereused for training the ANN classifier, and 30% (298,244 cases) for testing the classifier’s predictions. The ANN classifier algo-rithm for the multilayer perceptron model converged on two different activation functions for the classification training task,using a best estimation choice rule [84]. Specifically, the hyperbolic tangent activation function was used for estimating thecoefficients (synaptic weights) of the hidden layers in the neural network, and the Softmax activation function was used forestimating the synaptic weights of the output layer. Across the training and testing model classification tasks the best esti-mator for the error function was the cross-entropy method. The estimated network parameters are shown in the followingFig. 11. Positive larger values of the synaptic activation weights are associated with increased influence of the input layersubcategory or increased influence to the output layer subcategory corresponding to the incoming or outgoing synaptic con-nection of the hidden layers to inputs or outputs, respectively. Similarly, negative smaller values of the synaptic activation

Fig. 11. Estimated marginal means of the maximum network degree by simulation step across different levels of reorganization and severity of collapse inthe semantic network data ensemble.


weights are associated with the reversing of the influence strength to and from the corresponding input (independent vari-ables) or output (dependent variable) layers, respectively.

The profiles of the estimated synaptic activation weights indicate that the strength of the classification varies significantlyacross the nodes of the hidden layer. As can be seen in Fig. 11, hidden nodes H (1:4,5,6,9) show relatively stronger activationstrength than H (1:1,2,3,7,8). For example, from all the activation weight patterns, H (1:6) has very strong positive weightsfor input conditions characterized by a pattern (t, c, s, r) = (21–40, <1.43, 0.1, 1.0). The same hidden node, exhibits very strongpositive synaptic activation weights for output conditions where max(d) P 10 (and relatively low negative weights for allother conditions). In this example, the rule that the classifier imposes is to enhance the probability of observing a relativehigh maximum degree in the semantic network when relative infrequent and minor collapses occur early on in the networkevolution and the reorganization level is very high. A similar rule (increasing the probability of max(d) P 10) is reinforced bythe H (1:4) but when collapses do not emerge in the first 20 simulation steps. If we are to examine the patterns of conditionsfor which higher maximum degrees are enabled in the evolution of semantic networks, we need to look at which combina-tions of patterns the synaptic activation weights for the output layer max(d) P 10 are positive or negative. The last subgraphof the bottom panel in Fig. 11 provides the answer to this question. Classification patterns captured by hidden layers H (1:1,2, 4, 5, 6) yield Pr (max(d) P 10) > 0, whilst classification patterns H (1:3, 7, 8, 9) yield Pr (max(d) P 10) < 0.

A more comprehensive picture of the effect of reorganization patterns to the predicted patterns of centrality in thesemantic networks following collapses is shown in the following Fig. 12. The y-axis in the graph shows the predictedpseudo-probabilities of the MLP model classifier for the max(d) P 10 output category. The x-axis records the binned networkevolution over the 100 simulation steps of the data ensemble. The lines on the panel graph show the variability in the pre-dicted probability for achieving a nodal degree as high as 10 in the simulated network data ensemble, for different reorga-nization levels. The lines are paneled by severity levels (columns) and frequency of collapse incidents (rows). The structure ofthe MLP classifier ensures that the pseudo-probabilities across all categories of the output layer sum-up to 1.0, therefore,exceeding a probability of 0.5 for a single class (e.g., Pr(max(d) P 10) > 0.5) ensures the classifiers’ selection, and thus, itis the most probable outcome of the simulation. The results clearly show specific conditions under which different reorga-nization patterns in the network contribute to more central and cohesive semantic social networks.

More specifically, a number of interesting inferences can be drawn from these results:

(a) When the incidental frequency of collapse is very low (=1), that is, in conditions where collapses happen once duringthe simulation span, the network always achieves high centrality as can be seen from the first panel row in Fig. 12.Some exceptions occur when the severity of the collapse is high to very high (>.8) and the reorganization level is verylow (=0.1) where the probability asymptotically reaches 0.5 but does not exceeds that level.

Fig. 12. Synaptic activation weight profiles for the five hidden layers of the MLP classification model.


(b) When the severity level of the collapse incidents is low the semantic centrality of the emerging networks is notaffected significantly by them, but the outcome is highly sensitive to the level of reorganization prevalent in the sys-tem. The first column of the panel graph in Fig. 12 shows how different reorganization levels diverge over time on theirachieving level of semantic centrality. A case of specific interest is the third subgraph of the first column, where thecollapse frequency incident rate is 2.5. In terms of the specific simulation conditions this means that collapses emerg-ing every 100/2.5 = 60 simulation steps. The line patterns show how the effect of a collapse affects the network cen-trality temporarily, but when the reorganization level is relatively high, the network bounces back and achieves veryhigh centrality levels, even higher than in any other parametric combination of conditions.

(c) In general, at severity levels >0.1 when the frequency of collapse is moderate and above (incidence rate > 1.43), thenetwork centrality experiences long and short term collapses. Whilst some local level resilient patterns can be expe-rienced in the second and third column of the panel graph, the network experiences system-wide collapses at veryfrequent incident rates (=10). At moderate incident rates, the prevalence of resilience is sensitive to the level of reor-ganization of the system.

The overall neural network performance for the MLP classifier is shown in Fig. 13. Following the training and testing ofthe model, measures of the classifiers’ performance are computed and displayed in the graphs shown in Fig. 14. Specifically, areceiver’s operating characteristic (ROC) curve is plotted for the four predicted classification categories of the dependent var-iable (maximum nodal degree), and the area under the curve (AROC) as well as variable importance statistics computed asmeasures of the performance of the classifier.

The estimated and predicted results for the AROC of the MLP classifier (Table 5) indicate that for all the classification cat-egories of the dependent variable the AROC is very high (>.8). The high value of AROC signifies a departure from classificationpatterns expected by chance (diagonal reference line of the ROC curve). Furthermore, as can be seen also in the lift plot in

Fig. 14. Results of the ANN MLP Classification algorithm for the network simulation ensemble.

Table 5Area under the curve (AROC) statistics for the MLP classification.

Measured statistic Variable AROC Importance Normalized importance (%)

Dependent Max nodal 6 4 .914degree 5–6 .809

7–9 .80510+ .890

Independent Reorganization level .116 35Severity level .246 75Frequency of collapse .309 94Simulation step (binned) .329 100

Fig. 13. MLP predicted pseudo-probabilities for high semantic nodal degree centrality (dependent variable) under different alternative conditions of theindependent model parameters.


Table 6A summary of the inputs and output of the study results by analysis stage and type.

Stage Factorial input Factorial output

Semantic networkextraction

Natural language text from qualitative discourse andinterview transcribed texts

Initial activation weights based on co-occurrence frequencies andlinguistic weights based on extracted 3-g

ANN semanticknowledgerepresentation

Semantic activation weights from previous step Trained optimized semantic weights for semantic network

Agent-basedensemblesimulation

Semantic network structure (#nodes, #links,centrality, preferential attachment based on powerlaw distribution)

10,000 + Simulated semantic network ensembles based on time-steps, severity of collapse, frequency of collapse and self-organizationpotential

Longitudinal panelvariable analysis

Semantic network ensembles from stage 3 (t = [1,100]; s = [0, 1]; r = [0, 1]; c = [10, 100])

Panel factor statistical estimation using a GEE population-averagemodel selection

GLM estimation Semantic network ensembles from stage 3 (t = [1,100]; s = [0, 1]; r = [0, 1]; c = [10, 100])

GLM univariate model analysis for estimating marginal effects ofmean ensemble panel parameters

MLP classifier Semantic network ensembles from stage 3 (t = [1,100]; s = [0, 1]; r = [0, 1]; c = [10, 100])

simulation ensemble pattern estimation, and classification accuracyestimation of the MLP classifier


Fig. 13, and the very high AROC values for the two extreme maximum degree classes (where max(d) < 4 or max(d) > 10), theMLP classification performs very well for the top 50–70% of the cases classified.

In summary, the following Table 6 encapsulates all the input factors and outputs of the different analysis stages used inthis analysis, demonstrating a true multi and cross-disciplinary and holistic approach to understanding the complexity ofinteractions in large computational social science simulation ensembles. Our analysis aims to demonstrate the inferentialpower of combining a long tradition of computational social sciences and simulation ensembles framework [15,58] witha robust statistical estimation that enhances our ability to enhance the causal nature [65,66] of our estimations.

4. Discussion

4.1. Important analysis findings

The analysis performed on the simulated ensembles of network data allowed us to answer a number of critical questionsregarding patterns of self-organization in networks of semantic collective knowledge representation. We demonstrated thecentral role of scale-free distributions in allowing the emergence and propagation of knowledge patterns at the collectivesocial level. We also provided additional robustness in the empirical results obtained via our case studies, as well as inde-pendent validation of key emerging results involving the existence of tipping-points and scalability of collective semanticsocial structures. Throughout the statistical analysis of this paper we are able to raise a number of important points.

Firstly, we showed that collapses without the ability to reorganize have significant fragmentation effects on semantic net-work structure. Social change involving removal of key collective social institutions without examining the adaptive abilityof the remaining social structures can lead to wider fragmentation of social knowledge with highly unpredictable effects.Policy interventions aimed on affecting central social structures should take into account the ability of social actors and insti-tutions to adapt to changes. Social self-organization could be feasible but not without state transitions.

Secondly, sequential collapses on such systems can be quite volatile and unpredictable. It often leads either to total loss ofsemantic representation, or to reconstruction of the knowledge and re-emergence of scale free distributions. The bimodalityof those interactions is sensitive on the balance levels of parametric mixes. Limited evidence of the presence of nonlinearthresholds and tipping points of the interactions between collapse severity and self-organizing potential of the network ac-tors has been provided throughout this analysis. Under certain conditions, especially when repeated collapses occur, seman-tic structures exhibit bifurcations and easily shift (or oscillate) across possible outcomes.

Thirdly the potential for self-organization of semantic knowledge often has undesired effects, as it results in over-clusteringof knowledge flows and less cohesive social semantic structures that achieve limited collective social outcomes. The latterrelates to findings emerging from connectionism theory of knowledge affirming the potential to semantically activate knowl-edge in socially meaningful ways [25,85].

Going beyond the technical characteristics of the semantic network analysis provided in this paper, some broader meth-odological issues require our attention. Computational approaches to semantic network classification and analysis usingqualitative methodologies and psycholinguistic techniques have not been widely adopted and tested insofar. Computationalsocial science needs to discover and explore further the real power that the bridging of semantic knowledge representationwith qualitative social methods used in sociology, anthropology social psychology and other domains involved in the mod-eling of human dimensions. This can lead to enhanced understanding the complexity of interactions in coupled natural-anthropogenic systems especially in the face of broader social and environmental challenges for the future.

Our study shows that we can recreate semantic representations of knowledge capable of emulating semantic emergencematching our empirical findings. Nevertheless we lack further empirical findings from real-world participatory settingscapable to form a core evidence-base. Our scientific ability to disentangle theoretical and epistemological frameworks ofcollective semantic knowledge depends on the availability of additional future empirical settings and experiments.


4.2. Inferences for guided self-organization

Semantic network analysis overlaps with the aims of guided self-organization as a disciplinary scientific domain in morethan one ways. Elements of guidance in self-organizing patterns are present in the form of some of the algorithms used inthis study. Training and recursive estimation of the auto-associative memory systems performed by the Hopfield ANNs forcalibrating weights of semantic classification falls within the domain of simple self-organizing systems [37,64,74]. Extractedlinguistic concepts self-organize their knowledge flows into broader clusters or semantic categories based on objective infor-mation-theoretic optimization [30,56,95] of spreading-activation thresholds. To the extent to which such categorizations re-flect wider collective knowledge dynamics of social systems, semantic self-organization provides the seeds aiding andimproving our understanding of the micro-to-macro link between individual cognitive interactions on the one hand, andsocietal or institutional interactions on the other [79].

The proposed methodology for studying knowledge flows and knowledge representation at varying social scales ofsemantic inference is not a product of deterministic social engineering at the theoretical level. It rather emerges simply frommining a set of natural language textual data sources. In this sense, any patterns of emergence and knowledge organizationare not designed a priori. The latter comes to play in the ‘‘design versus organization’’ discussion [40,70], supported byempirical evidence in the domains of linked social-ecological systems [1,47]. The network complexity observed in collectivesemantic knowledge flows both in our empirical findings (Anmatjere semantic network) and our simulated ensembles sup-port the proposition that there exists a level of self-similarity in patterns and structure of semantic organization [83]. Pat-terns of structural self-organization such as the scale-free network distributions and the small-world effects observed in thisanalysis, as well as patterns of temporal self-organization (e.g., collapses-reorganization cycles) across network evolution,are all driven by changes in the informational context of the semantic context.

Nevertheless, guided evolution of such self-organizing knowledge systems not always result in lower entropic outcomes(or such of higher informational contexts). Phase transitions and regime shifts are neither uncommon nor improbable out-comes of guided self-organization. Our analysis explored sets of conditions under which semantic knowledge networks fallin states with lower informational context or rapidly collapse. A characteristic example stemming from empirical work is theeffect of government interventions in culturally sensitive settings such as small and remote Aboriginal communities in Aus-tralia. One of the unintended and high-risk potential outcomes from collapsing collective social structures related to com-munity livelihoods is that those same structures in turn support other collective institutions (such as Aboriginal culturalknowledge, family, and community support structures) that can become fragmented or even distinct over time. In the bestcase, they experience lack of resilience to adapt to future changes, a fact that the history of lost Aboriginal knowledge andculture across Australia should remind us.

Acknowledgments

The authors wish to acknowledge research contributions from Jocelyn Davies, Paul Box and Hannah Hueneke for theAnmatjere case study field work and analysis.

This research was supported by funding from Commonwealth Scientific and Industrial Research Organization (CSIRO) –Sustainable Regional Development/Indigenous Livelihoods Focal Research Area Project. Additional partial funding was pro-vided by the Australian Government Cooperative Research Centre Program through the Desert Knowledge CRC. Supplemen-tary funding for the main author has been provided by the NSF Virgin Islands Experimental Program to StimulateCompetitive Research (VI-EPSCoR), award no. 0814417. The views expressed herein do not necessarily represent the viewsof CSIRO nor Desert Knowledge CRC or its Participants. Willing engagement in the research by a wide range of people wholive and/or work in the region is also gratefully acknowledged.

References

[1] N. Abel, D.H.M. Cumming, J.M. Anderies, Collapse and reorganization in social-ecological systems: questions, some ideas, and policy implications,Ecology and Society 11 (2006) 17.

[2] A. Agresti, Analyzing repeated categorical response data, in: Categorical Data Analysis, John Wiley & Sons – Wiley InterScience, 2003, pp. 455–490.[3] P. Alacs, Complexity and uncertainty in the forecasting of complex social systems, Interdiscliplinary Description of Complex Systems 2 (2004) 88–94.[4] J.S. Albus, A model of computation and representation in the brain, Information Sciences 180 (2010) 1519–1554.[5] K. Alexandridis, Y. Maru, J. Davies, P. Box, H. Hueneke, Constructing semantic knowledge networks from the ground up: livelihoods and employment

outcomes in Anmatjere region, central Australia, in: R.S. Anderssen, R.D. Braddock, L.T.H. Newham, (Eds.), 18th World IMACS Congress andMODSIM09 International Congress on Modelling and Simulation, Modelling and Simulation Society of Australia and New Zealand and InternationalAssociation for Mathematics and Computers in Simulation, Cairns, Australia, 13–17 July 2009, 2009, pp. 2819–2825.

[6] K. Alexandridis, K. Coe, S. Garnett, Semantic analysis of natural language processing in a study of nurse mobility in the northern territory, Australia,Journal of Population Research 27 (2010) 15–42.

[7] K.T. Alexandridis, Exploring complex dynamics in multi agent-based intelligent systems: theoretical and experimental approaches using the multiagent-based behavioral economic landscape (MABEL) model, Ph.D. Dissertation, Purdue University, West Lafayette, Indiana. ProQuest Informationand Learning Company, UMI Dissertation Services (Source: DAI-B 67/09, p. 4949, March 2007), 2006, pp. 261.

[8] J.R. Anderson, A spreading activation theory of memory, Journal of Verbal Learning and Verbal Behavior 22 (1983) 261–295.[9] J.R. Anderson, Retrieval of information from long-term memory, Science 220 (1983) 25–30.

[10] T. Andreasen, P. Anker Jensen, J. Fischer Nilsson, P. Paggio, B. Sandford Pedersen, H. Erdman Thomsen, Content-based text querying with ontologicaldescriptors, Data and Knowledge Engineering 48 (2004) 199–219.


[11] L. Antiqueira, O.N. Oliveira Jr, L.d.F. Costa, M.d.G.V. Nunes, A complex network approach to text summarization, Information Sciences 179 (2009) 584–599.

[12] M.E. Bales, S.B. Johnson, Graph theoretic modeling of large-scale semantic networks, Journal of Biomedical Informatics 39 (2006) 451–464.[13] A.-L. Barabási, R. Albert, Emergence of scaling in random networks, Science 286 (1999) 509–512.[14] A.-L. Barabási, Linked: how everything is connected to everything else and what it means for business, in: Science and Everyday Life, Plume Books,

Penguin, New York, 2003.[15] S. Barton, V. Dohnal, J. Sedmidubsky, P. Zezula, Toward self-organizing search systems, in: A. Abraham, A.-E. Hassanien, V. Snasel (Eds.),

Computational Social Network Analysis: Trends, Tools and Research Advances, Springer, London, 2010, pp. 49–80.[16] P.M. Blau, A theory of social integration, The American Journal of Sociology 65 (1960) 545–556.[17] R. Blutner, Nonmonotonic inferences and neural networks, Synthese 142 (2004) 143–174.[18] R. Boyd, The puzzle of human sociality, Science 314 (2006) 1555–1556.[19] R.I. Brafman, Y. Dimopoulos, Extended semantics and optimization algorithms for cp-networks, Computational Intelligence 20 (2004) 218–245.[20] C. Brewster, K. O’Hara, Knowledge representation with ontologies: present challenges-future possibilities, International Journal of Human–Computer

Studies 65 (2007) 563–568.[21] E.J. Briscoe, A. Copestake, V. De Paiva, Inheritance, Defaults, and the Lexicon, Cambridge University Press, Cambridge; New York, 1993.[22] J.R. Buchler, H.E. Kandrup, Nonlinear Signal and Image Analysis, New York Academy of Sciences, New York, NY, 1997.[23] K.M. Carley, Computational organization science: a new frontier, PNAS 99 (2002) 7257–7262.[24] N. Chater, M.H. Christiansen, Computational models of psycholinguistics, in: R. Sun (Ed.), The Cambridge Handbook of Computational Psychology,

Cambridge University Press, Cambridge; New York, 2008, pp. 477–504.[25] M.H. Christiansen, N. Chater, Connectionist psycholinguistics: capturing the empirical data, Trends in Cognitive Sciences 5 (2001) 82–88.[26] A.M. Collins, E.F. Loftus, A spreading-activation theory of semantic processing, Psychological Review 82 (1975) 407–428.[27] G.S. Cree, K. McRae, C. McNorgan, An attractor model of lexical conceptual processing: simulating semantic priming, Cognitive Science 23 (1999) 371–

414.[28] G. Daqi, J. Yan, Classification methodologies of multilayer perceptrons with sigmoid activation functions, Pattern Recognition 38 (2005) 1469–1482.[29] W. De Nooy, A. Mrvar, V. Batagelj, Exploratory Social Network Analysis with Pajek, Cambridge University Press, New York, 2005.[30] A.R. de Soto, A hierarchical model of a linguistic variable, Information Sciences 181 (2011) 4394–4408.[31] S. Dennis, An unsupervised method for the extraction of propositional information from text, Proceedings of the National Academy of Sciences of the

United States of America 101 (2004) 5206–5213.[32] P. Dittrich, T. Kron, W. Banzhaf, On the scalability of social order: modeling the problem of double and multi contingency following luhmann, Journal

of Artificial Societies and Social Simulation 6 (2003). <http://jasss.soc.surrey.ac.uk/6/1/3.html>.[33] M. Drennan, The human science of simulation: a robust hermeneutics for artificial societies, Journal of Artificial Societies and Social Simulation 8

(2005). <http://jasss.soc.surrey.ac.uk/8/1/3.html>.[34] C. Eliasmith, Computation and dynamical models of mind, Minds and Machines 7 (1997) 531–541.[35] J.M. Epstein, Agent-based computational models and generative social science, in: J.M. Epstein (Ed.), Generative Social Science, Princeton University

Press, 2006, pp. 4–46.[36] M.H. Erdem, Y. Ozturk, A new family of multivalued networks, Neural Networks 9 (1996) 979–989.[37] R. Fujie, T. Odagaki, Self organization of social hierarchy and clusters in a challenging society with free random walks, Physica A: Statistical Mechanics

and its Applications 389 (2010) 1471–1479.[38] B. Galitsky, J.L. de la Rosa, Concept-based learning of human behavior for customer relationship management, Information Sciences 181 (2011) 2016–

2035.[39] B. Gawronski, F. Strack, On the propositional nature of cognitive consistency: dissonance changes explicit, but not implicit attitudes, Journal of

Experimental Social Psychology 40 (2004) 535–542.[40] C. Gershenson, F. Heylighen, When can we call a system self-organizing, in: W. Banzhaf, J. Ziegler, T. Christaller, P. Dittrich, J. Kim (Eds.), Advances in

Artificial Life, Springer, Berlin / Heidelberg, 2003, pp. 606–614.[41] C.D. Godsil, G. Royle, Algebraic Graph Theory, Springer, New York, 2001.[42] P. Goetz, D. Walters, The dynamics of recurrent behavior networks, Adaptive Behavior 6 (1997) 247–283.[43] C. Goldspink, Methodological implications of complex systems approaches to sociality: simulation as a foundation for knowledge, Journal of Artificial

Societies and Social Simulation 5 (2002).[44] J. Goldstein, Emergence as a construct: history and issues, Emergence 1 (1999) 49–72.[45] D.S. Gonzalez, A.V. Castro, Comparing Neural Networks and Efficiency Techniques in Non-Linear Production Functions, in: Universidad Complutense

de Madrid, Facultad de Ciencias Econmicas y Empresariales, Madrid, Spain, 2002.[46] M.S. Granovetter, The strength of weak ties, The American Journal of Sociology 78 (1973) 1360–1380.[47] L.H. Gunderson, L. Pritchard, Resilience and the Behavior of Large-Scale Systems, Island Press, Washington, DC, 2002.[48] Y. Guo, Z. Shao, N. Hua, A cognitive interactionist sentence parser with simple recurrent networks, Information Sciences 180 (2010) 4695–4705.[49] P. Guyot, S. Honiden, Agent-based participatory simulations: merging multi-agent systems and role-playing games, Journal of Artificial Societies and

Social Simulation 9 (2006) art. 8.[50] B. Hammer, A. Micheli, A. Sperduti, M. Strickert, Recursive self-organizing network models, Neural Networks 17 (2004) 1061–1085.[51] J.W. Hardin, J. Hilbe, Generalized Estimating Equations, Chapman & Hall/CRC, Boca Raton, Fla., 2003.[52] B. Horwitz, J.F. Smith, A link between neuroscience and informatics: large-scale modeling of memory processes, Methods 44 (2008) 338–347.[53] N.P. Hummon, T.J. Fararo, Actors and networks as objects, Social Networks 17 (1995) 1–26.[54] N. Kasabov, Adaptive modeling and discovery in bioinformatics: the evolving connectionist approach, International Journal of Intelligent Systems 23

(2008) 545–555.[55] M. Kimura, K. Saito, N. Ueda, Modeling of growing networks with directional attachment and communities, Neural Networks 17 (2004) 975–988.[56] T.-T. Kuo, S.-D. Lin, Learning-based concept-hierarchy refinement through exploiting topology, content and social information, Information Sciences

181 (2011) 2512–2528.[57] Z. Lei, B.C. Pijanowski, K.T. Alexandridis, Distributed modeling architecture of a multi agent-based behavioral economic landscape (MABEL) model,

Simulation: Transactions of the Society for Modeling & Simulation International 81 (2005) 503–515.[58] R.J. Lempert, S.W. Popper, S.C. Bankes, Shaping the next one hundred years: new methods for quantitative, Long-Term Policy Analysis, Rand

Corporation, The Rand Pardee Center, Santa Monica, CA, 2003.[59] M. Light, W. Greiff, Statistical models for the induction and use of selectional preferences, Cognitive Science 26 (2002) 269–281.[60] J.E. Lovelock, Gaia: A New Look at Life on Earth, Oxford University Press, Oxford, UK, 2000.[61] T.A.S. Masutti, L.N. de Castro, A self-organizing neural network using ideas from the immune system to solve the traveling salesman problem,

Information Sciences 179 (2009) 1454–1468.[62] P. Mika, Ontologies are us: a unified model of social networks and semantics, Web Semantics: Science, Services and Agents on the World Wide Web 5

(2007) 5–15.[63] NetLogo, A cross-platform multi-agent programmable modelling environment, in: Uri Wilensky and Northwestern University, Center for Connected

Learning (CCL) and Computer-Based Modelling, 2008.[64] B. Ning, X. Yu, J.-L. Hou, J. Zhao, Self-organization of directed networks through asymmetric coupling, Physics Letters A 374 (2010) 3739–3744.[65] J. Pearl, Causality: Models, Reasoning, and Inference, Cambridge University Press, Cambridge, UK, New York, 2000.

http://jasss.soc.surrey.ac.uk/6/1/3.html

http://jasss.soc.surrey.ac.uk/8/1/3.html


[66] J. Pearl, Causal inference in statistics: an overview, in: University of California, Los Angeles (UCLA), Computer Science Department, 2009, p. 51.[67] S.V. Pemmaraju, S.S. Skiena, Computational Discrete Mathematics: Combinatorics and Graph Theory with Mathematica, Cambridge University Press,

Cambridge, UK, New York, 2003.[68] I. Prigogine, I. Stengers, Order Out of Chaos: Man’s New Dialogue with Nature, Bantam Books, Toronto, New York, NY, 1984.[69] I. Prigogine, M. Sanglier, S. Instituts, Laws of nature and human conduct: specificities and unifying themes, Task Force of Research Information and

Study on Science, Brussels, 1987.[70] M. Prokopenko, Design vs. self-organization, in: M. Prokopenko (Ed.), Advances in Applied Self-Organizing Systems, Springer-Verlag, London, UK,

2007, pp. 3–17.[71] Y. Qian, C. Dang, J. Liang, D. Tang, Set-valued ordered information systems, Information Sciences 179 (2009) 2809–2832.[72] M.R. Quillian, Word concepts: a theory and simulation of some basic semantic capabilities, Behavioral Science 12 (1967) 410–430.[73] E. Ravasz, A. Barabasi, Hierarchical organization in complex networks, Physical Review E 67 (2003) 7.[74] H. Reuter, F. Jopp, J.M. Blanco-Moreno, C. Damgaard, Y. Matsinos, D.L. DeAngelis, Ecological hierarchies and self-organisation – pattern analysis,

modelling and process integration across scales, Basic and Applied Ecology 11 (2010) 572–581.[75] W. Rödder, F. Kulmann, Recall and reasoning-an information theoretical model of cognitive processes, Information Sciences 176 (2006) 2439–2466.[76] D.L. Sallach, Social theory and agent architectures: prospective issues in rapid-discovery social science, Social Science Computer Review 21 (2003)

179–195.[77] R.K. Sawyer, Social explanation and computational simulation, Philosophical Explorations 7 (2004) 219–231.[78] R.K. Sawyer, Social Emergence: Societies as Complex Systems, Cambridge University Press, 2005.[79] M.C. Schut, On model design for simulation of collective intelligence, Information Sciences 180 (2010) 132–155.[80] E. Serrano, A. Quirin, J. Botia, O. Cordón, Debugging complex software systems by means of pathfinder networks, Information Sciences 180 (2010)

561–583.[81] P. Shachaf, Social reference: toward a unifying theory, Library & Information Science Research 32 (2010) 66–76.[82] G. Siemens, Connectivism: a learning theory for the digital age, International Journal of Instructional Technology and Distance Learning 2 (2005).

<http://www.itdl.org/Journal/Jan_05/article01.htm>.[83] C. Song, S. Havlin, H.A. Makse, Self-similarity of complex networks, Nature 433 (2005) 392–395.[84] SPSS Inc., SPSS for Windows, SPSS Inc., Chicago, IL, 2003.[85] A. Stephan, The dual role of ‘‘emergence’’ in the philosophy of mind and in cognitive science, Synthese 151 (2006) 485–498.[86] R. Sun, The Cambridge Handbook of Computational Psychology, Cambridge University Press, Cambridge, New York, 2008. p. 753.[87] P. Toohey, Last drinks: the impact of the northern territory intervention, Quarterly Essay (2008) 1–97.[88] W.H. van Atteveldt, Semantic Network Analysis: Techniques for Extracting, Representing, and Querying Media Content, BookSurge Publishing,

Charleston, SC, 2008.[89] R. Veerhoven, World Database of Happiness, Erasmus University Rotterdam, 2008.[90] S. Wasserman, K. Faust, Social Network Analysis: Methods and Applications, Cambridge University Press, Cambridge; New York, 1994.[91] D.J. Watts, Small Worlds: The Dynamics of Networks Between Order and Randomness, Princeton University Press, Princeton, NJ, 1999.[92] W. Wei, P. Barnaghi, A. Bargiela, Rational research model for ranking semantic entities, Information Sciences 181 (2011) 2823–2840.[93] World Bank, World Development Indicators 2008, 2008.[94] R. Xia, C. Zong, S. Li, Ensemble of feature sets and classification algorithms for sentiment classification, Information Sciences 181 (2011) 1138–1152.[95] L. Xie, Y.L. Yang, Z.Q. Liu, On the effectiveness of subwords for lexical cohesion based story segmentation of Chinese broadcast news, Information

Sciences 181 (2011) 2873–2891.[96] B. Xu, X. Liu, X. Liao, Global exponential stability of high order Hopfield type neural networks, Applied Mathematics and Computation 174 (2006) 98–

116.[97] R.R. Yager, A framework for reasoning with soft information, Information Sciences 180 (2010) 1390–1406.[98] C.C. Yang, C.P. Wei, K.W. Li, Cross-lingual thesaurus for multilingual knowledge management, Decision Support Systems 45 (2008) 596–605.[99] H.-T. Zheng, B.-Y. Kang, H.-G. Kim, Exploiting noun phrases and semantic relationships for text document clustering, Information Sciences 179 (2009)

2249–2262.[100] G. Zhou, L. Qian, J. Fan, Tree kernel-based semantic relation extraction with rich syntactic and semantic information, Information Sciences 180 (2010)

1313–1325.[101] L. Zhou, Ontology learning: state of the art and open issues, Information Technology and Management 8 (2007) 241–252.

http://www.itdl.org/Journal/Jan_05/article01.htm

Documents

Collapse and reorganization patterns of social … and reorganization patterns of social knowledge representation in evolving semantic networks Kostas Alexandridisa, , Yiheyis Marub