The social-dance: decomposing naturalistic dyadic interaction dynamics to the ‘micro-level’

Caroline. P. Whyatt Rutgers University, United States caroline.whyatt@rutgers.edu

Elizabeth. B. Torres Rutgers University, United States ebtorres@rci.rutgers.edu

ABSTRACT The ‘social dance’ is an implicit, yet vital, characteristic of dyadic interactions. Attempts to characterize this complex behavior have illustrated unconscious levels of content and temporal entrainment within artificial social contexts. Yet, when viewed in a naturalistic setting, this complex systems problem faces a number of methodological and theoretical challenges. Utilizing precise kinematic recordings while adopting the ‘micro-movement’ approach, cross coherence analysis and tenets of graph theory, this paper presents an analytical framework to characterize unfolding, nonlinear temporal exchange and entrainment across a social dyad. This framework is empirically demonstrated within a clinical domain of individuals with known social difficulties: Autism Spectrum Disorder (ASD). Results illustrate the ability for this objective methodology to quantify variability in social dynamics, and profile dyadic entrainment during naturalistic exchange—with no a priori constraints or limitations. Viewed within the context of a clinical assessment tool for ASD, results facilitate consideration of clinician impact on dyadic exchange, and point to a potential refinement of core tasks associated with such clinical batteries.

MOCO '17, June 28-30, 2017, London, United Kingdom © 2017 Association for Computing Machinery.

ACM ISBN 978-1-4503-5209-3/17/06�$15.00

http://dx.doi.org/10.1145/3077981.3078055

CSS CONCEPTS • Applied computing → Systems biology; • Mathematics of computing → Distribution functions

KEYWORDS Social Interaction, Dyadic Exchange, Autism Spectrum Disorder, Network Analysis

1 INTRODUCTION Social interactions are an inherent component of our existence—yet little is known about how we engage in this complex ‘dance’. Seminal work illustrated the multi-layered nature of this process—demonstrating interplay between macro- and micro-levels of dyadic content and temporal interdependence [e.g. 1]. While the former refers to the transfer of information and is easily quantified through observational techniques, the latter refers to the timing and synchronicity of interactions, and exists on a micro-level more difficult to quantify. This temporal interdependence is demonstrated in the area of ‘coordination dynamics’. This data driven approach aims to circumvent

limitations in observational techniques by utilizing recurrent analytical models, and illustrates joint-action temporal coordination across a dyad [2-4]. Viewed from a computational modeling perspective, inverse and forward models of motor control [e.g. 5] imply that members of the dyad use this ‘motor language’ to extrapolate information for prediction of action consequences, and thus social intention [5, 6]. Drawing on principles of motor imitation and the mirror neuron system (MNS)[7], this ‘motor language’ is considered a nativist quality. Yet, despite ego- and allo-centric spatial coding within the parietal lobe [8, 9], our ability to translate allo-centric levels of complex temporal information to engage in a ‘social dance’ remains unaccounted for by this top-down content perspective. Defined by end-goal characteristics, the role of kinematics within MNS remains unclear—providing a ‘vocabulary’ [10], rather than a feed-forward method of control. Therefore, how does a dyad acquire implicit temporal interdependence? Despite illustrating the prominence of temporal interdependence, studies within ‘coordination dynamics’ are limited in their ability to deconstruct the underlying mechanisms. Results are contingent upon the use of artificial experimental paradigms, or an external metronome (or ‘beat’), to simplify this complex systems problem [2, 3]. These include the adoption of paradigms that artificially enforce ‘social’ oscillatory actions to an external metronome, such as dyadic chair rocking [2] or pendulum swinging [4]. The pre-imposition of a top-down approach, whereby areas and behaviors of focus are predefined, artificially constrains social dynamics—imposing a priori assumptions—while also serving to negate the impact of hidden self-emergent social dynamics. Indeed, within naturalistic settings, such research reverts to the use of observational metrics; reliant on visual quantification of discrete ‘macro-level’ behavioral outcomes [3]. This level of identification neglects the role of implicit self-emerging temporal exchange, while imposing restrictions on the type of a priori behaviors or ‘regions of interest’ that will be identified for examination. Further, the analytical methods employed often enforce assumptions of normality in the adoption of linear models, the use of epoch data that artificially constrains the dynamics of naturally, continuously unfolding social exchange, and summative level group statistics rather than an individualized approach. ‘Social signal processing’ [11] has made strides to provide computationally driven models to address these inherent difficulties. Through the use of computational, automatic analogues, this area of research is reexamining the quality and quantity of interactions using sequential learning models—such as Hidden Markov Models and Conditional Random Fields—to

MOCO’2017, June 2017, London, England UK

predict the outcome metrics of social signals across a dyad. While comprehensive mathematical models and learning techniques dominate this field, results remain underpinned by algorithms that are calibrated via a priori assumptions and subjective observational coding metrics [e.g. 12, 13, 14]. Reliance on observational, psychological tools for the visual quantification of social dyadic behavior illustrates the challenge posed by this complex systems problem. Incorporating levels of coder bias and error, such metrics are subjective and focus on the isolation of discrete macro-level social behaviors. However, within naturalistic social interactions, the dyad can be viewed as a continuous nonlinear, dynamical complex system, containing high levels of uncertainty, redundancy and time delay that must find a stable pattern for exchange, and thus, accurate prediction. With no a priori information, this setting initially renders inverse and forward models incomplete. It is proposed that one’s own evolving social-motor profile acts as a platform to guide and ‘de-code’ interactions via kinesthetic re-afference [15], and the ‘hidden’ layers of self-evolving dyadic synergies can be identified. This paper extends the concept of automatically detecting self-emerging patterns of complex actions to the social domain. Importantly, no criteria are pre-imposed for the selection of behaviors; rather, inherent dynamic variability across a dyad is harnessed. Adopting the micro-movement approach (see Section 3.2) the stochastic signature of underlying variability within the kinematic signature is profiled across the duration of the session, and individualized statistics empirically estimated per participant—ensuring no assumptions of normality. Finally, to ensure the translation of such data driven analytical methods to informative metrics, a semi-structured clinical tool examining social interactions is employed.

2 EXPERIMENTAL DETAILS

2.1 Subject information Autism Spectrum Disorder (ASD) [16] is a developmental disorder characterized by two axes of symptomatology; including impaired social and communicative ability. In addition, evidence indicates the prevalence of persistent sensory-motor difficulties, at both macro- [17] and micro-levels [18], across the ASD spectrum. As such, ASD provides an opportunity to examine social dynamics —particularly at the micro-level of motor interdependence. This model is demonstrated with sample results from five participants with a confirmed diagnosis of idiopathic ASD, and five neurotypical children. Illustrative case studies are provided alongside information on each participant. Rutgers University Institutional Review Board approved the study in compliance with the Helsinki Act.

2.2 Instrumentation To provide a robust characterization of social-motor control, performance was considered at the level of observational standardized psychological metrics and underlying kinematics.

2.2.1 Controlled social environment The Autism Diagnostic Observation Schedule (2nd ed.; ADOS-2, [19]), is considered the ‘gold-standard’ assessment tool to facilitate a diagnosis of ASD. A semi-structured play-based platform, the ADOS-2 creates a controlled social environment in which to examine natural, spontaneous social responses. Through the use of social ‘presses’ an examiner codes an individual’s spontaneous social skills to assess the presence and severity of symptoms. The ADOS-2 was harnessed to provide a controlled, yet natural, social setting to deconstruct macro-level, clinically relevant, social exchange (see Fig. 1 A). To facilitate this deconstruction, the presentation (i.e. timing) of social ‘presses’ was tracked. Note: all children completed module 3 with one ‘research reliable’ (ADOS-2 training certification) member of the clinical team. This consistency facilitated sequential decomposition of social dynamics and comparisons to be drawn at the group level. See Fig. 5 for constituent tasks.

2.2.2. Sensory-motor measurements Twelve lightweight wireless inertial measurement units (IMUs; APDM-OPAL, Portland OR) were placed on key anatomical landmarks across both members of the dyad (see Fig. 1 A). Comprised of an accelerometer and gyroscope, the IMUs provided synchronized measurements of temperature, tri-axial acceleration and angular velocity. Sampling at 128Hz, data was stored onto internal HD cards through robust logging to ensure no dropout. Data was subsequently extracted via APDM Motion Studio interface. This data—collected synchronously from all sensors across the dyad—provided an objective signal to quantify underlying levels of social-motor control and dyadic exchange.

3 COMPUTATIONAL OUTLINE AND RESULTS

3.1 Sensorimotor signature Calibrated temperature and tri-axial acceleration were extracted from each sensor upon completion of the ADOS-2. The tri-axial values of acceleration were resolved into a single vector (see Fig. 1 B-C) – the magnitude of resultant acceleration: using the Euclidean norm (see Equation 1).

(Eq.1)

Calibrated temperature and associated acceleration data-points were collated minute-by-minute (128 frames x 60 seconds) for the extraction of ‘micro-movements’ [e.g. 18, 20]

3.2 The ‘micro-movement’ approach Mirroring the multi-layered approach to social interactions, kinematic research exists on a layered continuum of analysis. The ‘micro-movement’ perspective provides a fine-grain level of kinematic analysis that moves beyond traditional summative metrics, with demonstrated sensitivity to clinical axes of symptomatology and levels of internal control [18, 20] (see Fig. 1). Through this micro-level approach, the stochastic signature of continuous minute fluctuations across any time-series can be empirically profiled—note in this instance fluctuations in the

The social dance: decomposing dyadic interactions to the ‘micro-level’.

MOCO’2017, June 2017, London, England UK

amplitude of acceleration are examined (see Fig. 1). To achieve this level of precision, the time-series is first mean centered, before moments of maximal deviation (peaks) are isolated (see Fig. 1 D). These values are subsequently normalized (ranging 0-1) to control for allometric variation using Equation. 2.

(Eq.2)

Where NPeak denotes the normalized peak time-series value, PeakValue is the original value, and AverageValue is the calculated average value between two local minima (see Fig. 1 D). The resulting normalized waveform thus represents minute fluctuations in peak amplitude of acceleration over time (see Fig. 1 E)—a ‘spike-train’ coined ‘micro-movements’ [18]). This underlying signature of variability is treated as a time-series of independently identically distributed events, under the general rubric of a Poisson Random Process. The stochastic signature of accumulation is subsequently profiled by exploring a range of probability distributions (both additive and multiplicative). Specifically, data from the normalized waveform is gathered into a frequency histogram, and maximum likelihood estimation (MLE) applied to empirically estimate the probability distribution function (PDF) that best characterizes the data (with 95% confidence) – see Fig. 1 F. In this instance, and in line with previous work characterizing sensory-motor profiles from neuro-typical and clinical populations [e.g. 20], the continuous Gamma family of PDFs best characterize the data (see Fig. 1 F). Using this family, the gamma parameters are extracted, namely shape (a) and scale (b), and the four moments of the empirically derived PDF estimated at the individual level: mean, variance, skewness, and kurtosis. These metrics are subsequently used for a characterization of underlying variability, and to derive inter- and intra-group comparisons as viewed on the Gamma

parameter plane (see Fig. 1 G). Note, skewness <0 is indicative of a distribution with a tail to the right and vice versa. This personalized approach, with no a priori assumptions of normality within the data set, is a departure from traditional metrics—a move toward the precision psychiatry framework [20]. Endorsing the continuous recording of evolving bio-physiological rhythms, this method lends itself to the precise profiling of naturalistic social interaction that encompasses unexplored aspects of behavior that unfold beneath awareness.

3.3 Objectively profiling dyadic entrainment The process outlined in Fig. 1 is performed on each minute-by-minute (7680 frames) segment of data from across the ADOS-2 session. This normalized waveform of ‘micro-movements’ (Fig. 2 E) represents micro-level variability of the individual’s own kinematic (i.e. physiological) signature, which is harnessed to consider temporal interdependence and entrainment across the dyad. The index of the micro-movements (i.e. peak order) from the above process is superimposed onto original frames to recover temporal information (frame number). Subsequently, we shift to the frequency domain, and each of these underlying signals of micro-movements is subject to power spectral analyses. Pairwise, each signal is decomposed to their constituent frequencies—i.e. power spectral analysis—and levels of cross-coherence examined using Welch’s moving average method [21]. Specifically, the micro-movement waveforms are windowed into overlapping sections (see Fig 2. B), minimizing noise associated with the decomposition of non-stationary signals, Discrete Fourier Transform performed and magnitude of squared coherence, and associated frequency between signals extracted.

Figure 1: Schematic overview of the raw data to micro-movement pipeline to utilize tri-axial acceleration recorded during the administration of ADOS-2. Through this pipeline of initial data pre-processing, the stochastic signature, or rate of

change, of underlying fluctuations within the kinematic recording (i.e. ‘micro-movements’) are empirically estimated using the Gamma family of continuous distributions (see main text). Gamma moments aid further inference and interpretation (size of the marker denotes kurtosis). See www.carolinewhyatt.com/moco-17 for larger images; password MOCO17. ©C.P

Whyatt & E.B Torres

MOCO’2017, June 2017, London, England UK

At frequencies where maximum coherence is identified, signals are examined using cross spectrum analytical methods to extract the phase ‘lead’ and ‘lag’ of each (see Fig. 2 C-D). In line with signal conceptualization as cyclical, this is quantified as an angular metric. The maximum level of coherence, phase lead information and frequency are extracted for the objective identification—and quantification—of temporal evolving interdependence across the dyad.

3.3 The evolving social dance: a network To provide an informative overview of exchange, all dyadic information is constrained to consider ‘leading’ information (positive angle values)—as isolated via the cross coherence analysis—and subsequently represented in an adjacency matrix of a weighted directed graph (see Fig. 2 E). To represent the relation between sensor (i) and sensor (j), we adopt the convention that they are nodes of a network linked according to a weight (the phase angle value) for the frequency corresponding to the maximum pairwise coherence level. Under this convention of positive leads, matrix entry (i,j) means node i leads node j of the network. These adjacency matrices (see Fig. 2 E) provide an initial visualization of the maximum values of coherence between sensors, the phase lead angles, and the frequency at

which the two sensors are found to correlate (ranges from 0 – SamplingRate/2 i.e. 64Hz). This refined data can be visualized via a dynamically unfolding weighted directed network graph (see Fig. 2 F). Network connectivity tools are thus adopted to study network self-emerging structure, degree distributions, modularity and other entrainment metrics we tailored to this problem domain—providing insight into the dyadic exchange. Specifically, functionality of the (Central) brain connectivity toolbox [22], is adapted to profile (Peripheral) body networks of individuals operating in tandem. With each sensor worn on anatomical landmarks representing a ‘node’, and weighted directed links representing the lead-phase information, the current model can assess levels of connectivity; both as summative statistics and evolving synergies (see Fig. 2 E-F). The connectivity of individual nodes across the network is visualized via network lines (i.e. links), with the strength of these connections (incoming and outgoing links) reflected in node size. Levels of coherence between nodes are illustrated by edge color, and the direction of this link (leading information – who leads who) is reflected in the presence and direction of an arrowhead terminal.

Figure 2: Generation of objective metrics of evolving dyadic exchange. The ‘micro-movement’ time-series waveform from across the dyad is extracted (pairwise) for sensors using the convention sensor iÆj. Illustrative data from 1 minute of data (128 frames x 60 seconds) the clinician and a female with ASD (age: 13 yrs.). These signals are subsequently decomposed—

power spectrum density—to identify a correlated frequency common across both. Signal content at these correlated frequencies is examined to extract coherence estimates including lead=lag angle information. Such metrics are

subsequently visualized as a connectivity network—providing a concise method to examine levels of inner and cross–entrainment (see main text), and for further synthesis into tailored metrics to model the ‘social dance’. Note: Panel F – red

arrows represent lead information within the clinician, blue within the child and black is hidden interpersonal self-emerging coupled dynamics: a new layer to the field. ©C.P Whyatt & E.B Torres

Furthermore, link color is indicative of the type of connection—encompassing intra-person nodes (across the individual—red across participant, blue across clinician), or inter-person nodes across both individuals in the dyad (black links). This level of visualization provides initial insight into the type of synergies identified within the network. Thick linking lines are indicative of the connectivity strength—i.e. level of sustained lead information as derived via the phase angle value (see Fig. 2 F). Thus, heavy interwoven network lines between nodes explicitly represent prominent connections coming into or leaving out of a node (a hub in the network). Finally, modules within the network (nodes that are maximally interconnected forming a synergistic subnet and minimally connected with the rest of the network) are coded by node color (representing which module the node belongs to). This visualization facilitates a further understanding of how anatomical landmarks across each individual, and between individuals (intra- and inter-person), is being coordinated at a micro-level. Visualized every 60-seconds, this method provides a dynamic insight into the evolving metrics of social exchange. Sample results are demonstrated for a female participant with ASD aged 13 years old and a 10-year-old female control participant (see Fig. 3).

3.4 Can objective metrics inform psychological tools?

Coupling a ‘traditional’ observational tool with objective physiological metrics of social skills, this project provides a robust framework to consider dyadic exchange. The ADOS-2 provides a controlled social environment that encourages naturalistic exchange—with no external sources for content or temporal coupling. Furthermore, the ADOS-2 provides a structured environment with pre-defined semi-structured landmarks designed to illicit spontaneous social exchange. As such, the question is posed; can objective metrics of social exchange refine our conceptualization of social dynamics from a macro-level? Viewed within the broader context, such metrics may provide insight into the underpinnings of this psychological, observational tool. Specifically, despite a dyadic exchange, the ADOS-2 metrics are designed to consider the impact of the participant alone. Yet, drawing on leading and lagging levels of temporal exchange, the social outcomes can be decomposed to consider the impact of both parties, while also considering the impact of task. To this end, the objective dyadic metrics are utilized to blindly isolate elements of the ADOS-2 in which the clinician or participant are leading the exchange. In particular, the weight of total out-strength, phase leading information from across the network nodes is calculated across both members (see Fig. 4). This dynamic timeline provides a metric of the evolving synergy across the dyad. The illustrative example of Fig. 4 demonstrates the objective characterization of the clinician leading the exchange for a proportion of time across both the control and experimental sessions—an impact not accounted for by the original ADOS-2 (clinician coded) observational metrics. Moreover, when deconstructed to consider the impact of the clinician during sessions with control participants vs. participants with ASD, the leading impact of the

clinician varies substantially—with notably higher levels during sessions with individuals with ASD. Perhaps reflective of the known social difficulties—and/or underlying motor patterns associated with ASD that may prevent a full symbiosis between both members. This finding calls into question the ability of this psychological tool to fully (and objectively) quantify social dynamics.

Figure 3: Illustration of the evolving social dyad during the administration of ADOS-2 module 3. The top panel provides two ‘snap-shots’ of this social network during

minutes 22 and 25 of administration with a female participant with ASD. These frames coincide with the

event landmarks of ‘conversation & reporting’, and ‘emotion’ component. As illustrated, the clinician is found

to ‘lead’ this extract as denoted through the presence of directed (c.f. arrow head), strong (c.f. weight of line)

connections denoting the spontaneously self-emerging coupled dynamics of the dyad (black lines). The bottom

panel provides an illustration of the same event landmark occurring during the administration of module 3 with a

control participant. In this instance, the participant demonstrates a leading profile during minute 32, with

lower levels of connectivity found across the dyad during the more charged ‘emotion’ component at minute 33. These frames are provided on a minute-minute basis

across the ADOS-2. However, in line with the continuous and dynamic nature of social interactions, these are best

visualized as a movie format (see: http://www.carolinewhyatt.com/moco-17; use password

MOCO17 for access). ©C.P Whyatt & E.B Torres

MOCO’2017, June 2017, London, England UK

Through refining this information to consider the difference in signal strength between both members, moments of maximal deviation were isolated across the cohort. When viewed within the context of the ADOS-2 task landmarks for module 3, these coincided with the ‘emotion’ component and ‘telling-a-story’ task (see Fig. 5 A). To explore these components further, the physiological signatures recorded for each participant were isolated and examined in more detail (see Fig. 5). An unexpected level of variability, due to an increase in kinematic variation (micro-movements), characterizes the ‘emotion’ component for the ASD sub-sample (see Fig. 5). While the ‘emotion’ component entails no instructed movement, the IMUs registered a level of lower-temperature motion (a characteristic of spontaneous actions [23])—one statistically different from directed movements recorded during the ‘telling-a-story’ task. Fig. 5 provides an illustration of this contrast for an ASD participant (age 13 yrs.) and a control participant (age 10 yrs.). As this is examined from a dyadic perspective, the

clinician’s physiological signature is also profiled per session (in the corresponding lower panels). The ASD spread of Gamma moments on the parameter plane, particularly those denoted by circular markers (representing the ‘emotion’ component), clearly contrast with those of the control participant, but also of the examiner in both sessions. Indeed, the motor demands of the ‘telling-a-story’ task and the affect demands of the ‘emotion’ component are automatically uncovered in the stochastic profile of the micro-movements extracted from these bodily rhythms. Given the non-parametric nature of the data (i.e. no assumption of normality is being imposed), Wilcoxon ranked sum tests were completed to gain insight into the distribution of PDFs that best characterize participant data. As demonstrated in Table 1, visual trends were reflected in a statistically significant (p<0.05) difference in the characteristic properties of the ASD participants’ sensory-motor signature during completion of the ‘emotion’ vs. ‘telling-a-story’ task.

Figure 4: Illustrative example of leading information across the ADOS-2 administration. Information from the weighted directed graph is summarized to consider who is leading throughout the session. Specifically, leading information derived from a pairwise cross coherence analysis across the dyad is summed to result in node-out strength for each member across

the session. Four cases are presented (ASD participants aged 13 and 6 yrs.; control participants aged 10 and 7 yrs.) demonstrating a pattern for clinician dominance during clinical ASD exchange. The difference in out-strength between each member of the dyad is subsequently calculated (see inset), and quantified as percentage of time in the ‘lead’. This

methodology thus provides a summary of the state of the network across the dyadic exchange. ©C.P Whyatt & E.B Torres

Specifically, the mean variability in the underlying normalized (0-1) micro-movement profile for children with ASD was significantly lower for the ‘emotion’ component in relation to the control group, and the clinician. Due to the normalization process (see Equation 2), this lower normalized metric is indicative of higher levels of acceleration within the movement profile of individuals with ASD. This trend extends to significant differences between the ASD and control populations during administration of the ‘telling-a-story’ task (N=10 comparison).

Task Pair-wise comparison p-Value Wilcoxon Ranked Sum N=2 N=10

Emotion Component

ASD- Control 0.04* 9.313e-06* ASD-Clinician 0.05 0.01* Control-Clinician 0.81 0.02*

Telling-a-story ASD-Control 0.52 0.02* ASD-Clinician 0.11 0.09 Control-Clinician 0.10 0.11

Table 1: Wilcoxon Rank Sum examination of group characteristics.

These results raise the question—can objective physiological metrics tap into a level of affect that is currently beyond the scope of traditional psychological tools? As noted, the ‘emotion’ component contains a range of questions that are designed to target emotionally salient constructs. Although often stereotyped as experiencing difficulties with emotional

regulation, expression and empathy, such work may give voice to those feelings for individuals with ASD. Moreover, difficulties with social ‘connection’ at the macro- overt level (i.e. reporting of emotions) may be reflected in difficulties ‘leading’ this phase of the administration (see Fig. 3 & 4). Coupled with results indicating the presence of spontaneous sensory-motor variability during this component (see Fig. 5), such methods may provide novel insight into the role of this micro-level fluctuation as a coping mechanism and may speak to a level of empathy—or discomfort—beyond our current conceptualization.

4 DISCUSSION & CONCLUSIONS Through coupling psychological and physiological metrics, an embodied approach to social exchange has been illustrated. Drawing on principles of exchange, the underlying physiological motor signature has been continuously harnessed to quantify this evolving dynamic. In a stark departure from previous non-computational and computational methods that reverted to the use of observational tools, a priori assumptions and/or limitations to simplify this complex system, this work has demonstrated the ability to track subtle aspects of non-linear social exchange inherently present in natural situations. Drawing on tenets of graph theory, this methodology facilitates the identification and dynamic tracking of self-emerging dyadic synergies—rather than pre-imposing assumptions or the artificial creation of oscillatory exchange.

Figure 5: Sequential deconstruction of the ADOS-2 module 3 in light of objective physiological parameters. For clarity, data is visualized for two female participants (ASD: 13 yrs. and control 10 yrs.). Please see

http://www.carolinewhyatt.com/moco-17; use password MOCO17, for additional material summarizing sessions with 10 individuals (5 ASD and 5 Control). Timing of maximal points of difference in out strength connectivity were automatically

isolated across participants (see Fig. 4) and cross-referenced as corresponding to the ‘emotion’ and ‘telling-a-story’ tasks (A). Summative gamma parameters (see Fig. 1) of the underlying physiological signatures for the corresponding timestamps were profiled (B: ASD, C: Control). In order to further elucidate the result, corresponding maximum

temperature values were integrated (color coded markers), with higher temperatures indicative of active, overt actions. As demonstrated, participants with ASD show notable variation in the underlying sensorimotor signature. This is particularly

noted in the ‘emotion’ component (see main text). ©C.P Whyatt & E.B Torres

Through the individualist empirical estimation of statistical parameters, and a removal of assumptions of data normality, this framework considers the evolving noise-to-signal ratio and synchrony at an interpersonal and precise intrapersonal level. Building on principles of kinesthetic re-afference, this work thus provides a framework to consider the impact of the individual’s own kinematic signature in the levels of dyadic coupling—a move away from top-down theoretical models. From a clinical perspective, these objective metrics provide insight into founding assumptions for the design and creation of observational tools. Psychological batteries, such as the ADOS-2, that examine social exchange as a construct do so from a single perspective: that of the examiner. This restricted conceptualization is reflected in scoring metrics that presume minimal examiner influence—with no scope for leading. Yet, systematic examination of the underlying kinematic signature demonstrates the interplay of lead and lag information of the dynamically coupled dyad, in a conceptualization more akin to natural dyadic exchange. High levels of clinician leading during ASD sessions hint at both the struggle of the individual, but also the impact of the clinician on task outcome—an impact currently neglected in task design. In addition, the integration of objective kinematic analytics demonstrated the role of spontaneous actions—those beyond the observational remit of the instrument. Drawing on maximal moments of participant and clinician ‘leading’ information, pertinent ADOS-2 tasks were identified; namely the ‘emotion’ and ‘telling-a-story’ components. While the design and requirements of the ‘telling-a-story’ task denote a relatively demanding motoric activity, the ‘emotion’ component lacks this motor element. However, the level of uncertainty and emotional distress of the ‘emotion’ component may result in the different stochastic landscape for individuals with ASD. Continued reliance on traditional visual quantification of overt macro-level actions enforces an epoch approach, while negates fundamental micro-level fluctuations in dynamical temporal control that may provide insight into both underlying sensory-motor functioning and emotional state. As such, the methods presented here may facilitate an objective understanding of both the physiological and emotional state of individuals on the Autism Spectrum, supplement clinical expertise in the design and refinement of clinical tools, and serve as a method to dynamically track and profile axes of symptomatology. Viewed within the context of accessible wearable technology for M-health, and within the remit of ‘precision psychiatry’, results point toward a new approach to behavioral modeling; one that aims to open dialogue between research and the clinical field to enable transformation and re-conceptualization of precision phenotyping. By introducing this new framework for dynamic diagnoses and stochastic outcome measures for longitudinal tracking of somatic-motor change, we aim to bridge the current gap between descriptions of overt naturalistic behavior and genotypic data. As such, this personalized objective model has the potential to leave a lasting mark on the clinical arena.

ACKNOWLEDGMENTS We would like to thank the children and families that took part in this research, our undergraduate students and clinical team members A. Mars and B. Patel for their assistance. The New Jersey Governor’s Council for Medical Research and Treatment of Autism supported this work. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org.

REFERENCES [1] Kaye, K. and A. Fogel, The temporal structure of face-to-face communication

between mothers and infants. Developmental Psychology, 1980. 16(5): p. 454. [2] Richardson, M.J., et al., Rocking together: Dynamics of intentional and

unintentional interpersonal coordination. Human movement science, 2007. 26(6): p. 867-891.

[3] Schmidt, R., et al., Understanding social motor coordination. Human movement science, 2011. 30(5): p. 834-845.

[4] Schmidt, R., et al., Effects of social and physical variables on between-person visual coordination. Ecological Psychology, 1994. 6(3): p. 159-183.

[5] Wolpert, D.M. and M. Kawato, Multiple paired forward and inverse models for motor control. Neural networks, 1998. 11(7): p. 1317-1329.

[6] Frith, C.D. and D.M. Wolpert, The neuroscience of social interaction. 2003. [7] Williams, J.H., et al., Neural mechanisms of imitation and ‘mirror

neuron’functioning in autistic spectrum disorder. Neuropsychologia, 2006. 44(4): p. 610-621.

[8] Georgopoulos, A.P., A.B. Schwartz, and R.E. Kettner, Neuronal population coding of movement direction. Science, 1986. 233(4771): p. 1416-1419.

[9] Buneo, C.A., et al., Direct visuomotor transformations for reaching. Nature, 2002. 416(6881): p. 632-636.

[10] Rizzolatti, G., et al., Functional organization of inferior area 6 in the macaque monkey. Experimental brain research, 1988. 71(3): p. 491-507.

[11] Delaherche, E., et al., Interpersonal synchrony: A survey of evaluation methods across disciplines. IEEE Transactions on Affective Computing, 2012. 3(3): p. 349-365

[12] Avril, M., et al., Social signal processing for studying parent–infant interaction. Frontiers in psychology, 2014. 5.

[13] Saint-Georges, C., et al., Do parents recognize autistic deviant behavior long before diagnosis? Taking into account interaction using computational methods. PloS one, 2011. 6(7): p. e22393.

[14] Provenzi, L., et al., A dynamic system analysis of dyadic flexibility and stability across the Face-to-Face Still-Face procedure: application of the State Space Grid. Infant Behavior and Development, 2015. 38: p. 1-10.

[15] Holst, E., H. Mittelstaedt, and P. Dodwell, The principle of reafference: interactions between the central nervous system and the peripheral organs. Perceptual Processing: Stimulusequivalence and Pattern Recognition, 1971: p. 41-71.

[16] Association, A.P., Diagnostic and statistical manual of mental disorders (DSM-5®). 2013: American Psychiatric Pub.

[17] Whyatt, C. and C. Craig, Sensory-motor problems in Autism. Frontiers in integrative neuroscience, 2013. 7: p. 51.

[18] Torres, E.B., et al., Autism: the micro-movement perspective. 2013. [19] Lord, C., P.C. DiLavore, and K. Gotham, Autism diagnostic observation

schedule. 2012: Western Psychological Services Torrance, CA [20] Torres, E.B., et al., Toward Precision Psychiatry: Statistical Platform for the

Personalized Characterization of Natural Behaviors. Frontiers in neurology, 2016. 7

[21] Welch, P., The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Transactions on audio and electroacoustics, 1967. 15(2): p. 70-73.

[22] Rubinov, M. and O. Sporns, Complex network measures of brain connectivity: uses and interpretations. Neuroimage, 2010. 52(3): p. 1059-1069.

[23] Torres, E.B. and B. Lande, Objective and personalized longitudinal assessment of a pregnant patient with post severe brain trauma. Frontiers in human neuroscience, 2015. 9.