Embed Size (px)
Citation preview
Beyond WiresEditor: Cecilia Mascolo • [email protected]
60 Published by the IEEE Computer Society 1089-7801/12/$31.00 © 2012 IEEE IEEE INTERNET COMPUTING
T he emergence of smartphones as a viable sensing platform has radically shifted the landscape of mobile-sensing research.
Mainstream consumer smartphones equipped with a range of sensors are the everyday com-puting platform for millions of people world-wide. This has dramatically lowered the barrier to building and deploying large-scale sensing systems that can operate not only on a personal level but also at a community scale, sensing entire cities and beyond.1
Even years after early stage prototypes,2,3
researchers are still converging over how to design smartphone sensing systems. A key open question is precisely how to leverage these systems’ ability to sense beyond a single individual as they collect and interpret sensor data. In this direction, we’ve made steady progress in our ability to automatically extract and reason about complex social struc-ture and dynamics using data readily available to smartphone systems. For instance, macroscale mobility patterns and social network topologies can be mined from the Call Detail Records (CDRs) of mobile phones.4 At a more personal scale, coarse proximity data from bluetooth radios can distin-guish different types of social ties,5 or fine-grain characterizations of face-to-face interactions6
can be inferred from wearable sensors’ audio streams.
Yet the design of existing smartphone sens-ing systems still focuses most strongly on the individual, carefully considering issues such as usability or user engagement. This is entirely appropriate given the mobile phone platform’s highly personal nature, but such considerations also imply that people live in a vacuum, unaf-fected by others’ behavior. In fact, individuals are tightly connected via various social and physical processes. Phenomena including social influence,7 homophily (the tendency of similar people to form social bonds),8 and information diffusion9 collectively cause various types of interpersonal dependencies. Users form complex hierarchies of densely connected community groups and are influenced by group behavior and social networks. By neglecting the relation-ships and networks that link users, we’re ignor-ing an important part of the design space for smartphone sensing.
What if sensing systems comprehensively understood the complex community dynam-ics within a user population? How should they be designed and function given such aware-ness? This column’s underlying conjecture is that we can revolutionize smartphone sensing if it becomes community-aware, with systems empowered to see beyond just a single user and designed to understand and react to a set of rich
Community-Aware Smartphone Sensing SystemsNicholas D. Lane • Microsoft Research Asia
Despite people’s central role within smartphone sensing, such systems remain
largely oblivious to the effects of social networks and community dynamics.
How might smartphone sensing systems change if they could see more than
isolated individuals? What if sensing systems could not only understand these
social effects but could leverage them in their day-to-day operations — for
example, as they collect and interpret mobile sensor data?
IC-16-03-Bwrs.indd 60 4/2/12 10:40 AM
Community-Aware Smartphone Sensing Systems
MAY/JUNE 2012 61
and fluctuating connections within the wider community.10
To make this discussion more concrete, I look at three mobile-sensing scenarios. For each one, I consider how community-aware sensing can alter existing practice. Specifically, I examine how a community-aware approach can lead to radically new ways of sensing communities in a distributed manner, allow individ-ual users to assume more sophisti-cated roles within these systems, and enable sensor-based models of human activity to generalize to large user populations despite the presence of significant inter-user diversity.
Mobile HealthUsing mobile phones to continu-ously monitor fine-grained physical activity11 or a broader set of health outcomes could change how society diagnoses and treats medical con-ditions. Such mobile systems are rapidly advancing in the areas of sensing, inference, persuasion, and resource management. However, virtually all of them assess health using data from individual users. This ignores an important dimen-sion of personal health: how friends, family, and others influence users’ health and well-being. For example, monitoring exposure to communi-cable illnesses — critical to at-risk people such as the elderly or those with certain chronic conditions — has limited effectiveness if we con-sider only the individual. However, if systems understand the mobility patterns and activities of both indi-viduals and those they routinely interact with, we can combine such information with epidemic models that capture the aggregate implica-tions of social interactions. This will better equip sensing systems to track the risk factors of developing dis-eases (especially in relation to par-ticular behaviors). In support of this direction, researchers have recently demonstrated the effectiveness of
monitoring changes in socialization, such as face-to-face interactions, to act as an indicator for contagious diseases such as common colds and influenza.12
Similarly, personal interactions can significantly influence health-related behavior, such as an indi-vidual’s eating habits13 or mental state14 (see Figure 1). A system that understands the dynamics of such interactions — and the overall social network in which they occur — can have a much broader perspective on such aspects of well-being. By com-bining microscale, on-body sensor data with macroscale awareness of external social factors, mobile health systems will be better equipped to not only capture but also predict health outcomes.
Crowdsourcing and Human ComputationExploiting “people-in-the-loop” within mobile phone sensing systems is a natural direction given that a smart-phone is a personal device with which users frequently interact.
Earlier work has studied the poten-tial for leveraging crowdsourcing to intelligently collect mobile sen-sor data.15 Now, hybrid systems are emerging that blend the strength of human computation16 with machines to produce systems with otherwise unachievable operating points (such as trade-offs between accuracy and delay).17 However, the conven-tional view of these hybrid systems maintains a narrow perspective on people — that is, the systems are designed so that the same type of loop is used repeatedly for everyone and only one person is embedded in it.
Crowdsourcing achieves scale by creating millions of these individual loops. The approach, while undeni-ably powerful, misses crucial oppor-tunities by treating everyone as homogeneous cogs in a machine. In reality, a group’s suitability for per-forming crowdsourcing tasks (for example, collecting or processing data) varies from group to group, depending the skills and attributes of the individuals involved. What’s missing are automated methods to
Figure 1. The correlation between the social connections (graph edges) among people (nodes) and a categorization of daily food choices (node size) for a study conducted in Beijing, China. More generally, substantial evidence from medical and social science exists linking health outcomes to a variety of social phenomena. However, virtually none of the existing mobile health systems consider such connections when assessing users’ health.
IC-16-03-Bwrs.indd 61 4/2/12 10:40 AM
Beyond Wires
62 www.computer.org/internet/ IEEE INTERNET COMPUTING
identify and characterize user com-munities. Once available, this infor-mation can be used to tune a system’s operation and more easily maintain design requirements (such as accu-racy and cost).
For instance, for a crowdsourc-ing system to effectively perform a data interpretation task, users in the community must share char-acteristics such as location, knowl-edge, skills, and available free time. Other sensing systems require just the opposite and need diversity. One example is crowdsourced experi-ments in which users are the study participants (such as a user study), which normally require forming diverse groups representative of an entire population. Through careful, automated selection of user com-munities that perform those sensor-based crowdsourcing tasks (such as data collection or interpretation) they’re best suited for, people-in-the-loop systems can achieve higher quality and less variable results. Crowdsourcing systems unaware of a community’s structure will be sub-ject to unpredictable performance that depends on the composition of a
group of users who happen to accept an offered bundle of system tasks. Existing work in the context of matching individuals to roles within participatory sensing campaigns18 can likely assist in this area.
Activity RecognitionIn this final example, I consider a recently published activity-recognition framework, Community Similarity Networks (CSN),19 because it adopts community-aware principles to over-come challenges to large-scale mobile classification.
As a user population’s size increases, so does the amount of diversity it contains. Users can vary for sev-eral reasons, a clear example being physical differences such as height, weight, or gender. People can live and work in different places and have different cultures and socio-economic origins. Although they might do the same basic set of activi-ties (workout, go to work, socialize) they can do these activities in very different ways. Mobile systems com-monly use various supervised clas-sifiers (example-based) to recognize behavior and context. However, these
classifiers fail to generalize to such levels of diversity — leaving accu-racy unpredictable and dependent on precisely who happens to use the system.10,19
CSN approaches this problem by using different dimensions of inter-user similarity (that is, physical, lifestyle, and sensor-data-driven) to build a weighted user graph, a simi-larity network that captures the vari-ous ways users are alike (see Figure 2). Under CSN, each similarity network is embedded within a chain of classi-fier training algorithms to emphasize labeled training data between users who share common traits. As a result, CSN can personalize generic classi-fiers to each user’s different charac-teristics. Using a variety of similarity networks is necessary because no single type of similarity is effective across all the categories of activi-ties. For example, physical charac-teristics are useful for activities such as exercise, running, or climbing stairs. Lifestyle characteristics (such as occupation, temporal patterns, and where people live and work) are more beneficial to activit ies such as transportation-mode inference.
Figure 2. The Community Similarity Networks (CSN) framework. The framework trains personalized activity classifiers sensitive to the interpersonal differences and similarities among users. CSN incorporates various similarity dimensions (such as shared physical and lifestyle traits) enabling activity recognition to better generalize to diversity in the user population.19
Crowdsourcedsensor dataand labels
Physical similarity network
Lifestyle similarity network
Sensor data similarity network
Communitysimilarity
network-basedlearning
Personalized user model 1
Personalized user model n
RijRijRij
Yij MijYij Mij Mij
Yij {,}{,}
{,}
{,}{,}
{,}{,}
{,} {,}
Fi1, j1 Fin, jn
�
…
…Fi2, j2
RijRijRij
Yij MijYij Mij Mij
Yij {,}{,}
{,}
{,}{,}
{,}{,}
{,} {,}
Fi1, j1 Fin, jn
�
…
…
…Fi2, j2
IC-16-03-Bwrs.indd 62 4/2/12 10:40 AM
Community-Aware Smartphone Sensing Systems
MAY/JUNE 2012 63
CSN clearly demonstrates how com-munity awareness built into sensing systems at design time allows nat-urally occurring phenomena, such as communities with shared behav-ioral patterns, to overcome challeng-ing sensing problems.
Toward Community Awareness for Smartphone SensingCommunity-aware smartphone sens-ing systems will need a deep under-standing of the diverse ties that connect people and cause networks of communities to form within diverse user populations. Such sys-tems will leverage this understand-ing while performing basic sensing system operations to fundamentally change how these operations occur. Noticeably, the three examples pro-vided all use communities in very different ways. At this nascent stage of community-aware sensing, we aren’t yet close to a generalized framework for leveraging communi-ties within smartphone systems. The complexities and sheer variety of
community interactions and social ties within large-scale populations are extensive, leading us to ask, which of these interactions and ties are the most salient to the operation of sensing systems? Can we embed sufficient understanding of these effects into system components without resorting to developing sys-tems on a case-by-case basis?
Central to community-aware systems’ evolution will be the development of community models — computational models of the net-works that bind individuals — that can mine the various links between individuals, recognize different community types comprising tightly connected people, and determine these communities’ collective char-acteristics. Figure 3 illustrates two key roles these models will have within community-aware smartphone sensing systems — specifically, influ-encing how phenomena are sensed and guiding a variety of system oper-ations (such as data collection) and optimizations. An expanded under-standing of community dynamics
will alter how these systems sense real-world phenomena ranging from traffic patterns to user preferences toward places of interest20 or even smartphone application usage.21 Com-munity models will play a role in sensing whenever connections between people can affect the phenomena in question.
Similarly, we can embed commu-nity models into key sensing opera-tions to guide how these components function or optimize their perfor-mance. A clear illustration of this is CSN, in which the entire work-flow of training activity-recognition models changes to incorporate the similarity networks that exist in the user population. Another example is the sensing systems that leverage crowdsourcing discussed previously; in such systems, community models can intelligently optimize the selec-tion and assignment of communities to specific sensor data collection or interpretation tasks. By introducing the community model and leverag-ing its aggregate predictive power, we can enable sensing systems to
Figure 3. A community-aware smartphone sensing system. Central within these systems will be the community model, which captures the social structure within a user population and the communities that form. The model can both impact how the system senses phenomena and guide how various sensing system operations are performed and optimized.
Smartphone sensing client
Sensor data classi�ers
Sensors
Smartphone
GUI
Network interface
Communitymodel
Phenomena model
Storage
Cloud intrastructure
StorageTraining and
inferencefor classi�ers
Dataanalytics
Network interface
IC-16-03-Bwrs.indd 63 4/2/12 10:40 AM
Beyond Wires
64 www.computer.org/internet/ IEEE INTERNET COMPUTING
optimize common system trade-offs, such as cost, accuracy, energy, and latency, despite unpredictable user behavior.
T o progress toward general-purpose community-aware smartphone
sensing systems, we’ll need more sophisticated community models that will in turn demand new approaches to sensing at both the micro-(personal) and macro- (community) scale. Even more pressing is the need to find ways to seamlessly connect these two viewpoints in the design and architecture of large mobile-sensing systems, which would let us migrate from existing examples of community-aware sensing that fail to generalize beyond single narrow domains. Finally, privacy remains perhaps the most critical obstacle. The extensive awareness of com-munities envisioned, even if it were possible today, requires informa-tion that the general public would be uncomfortable sharing. We’ll need methods for either performing this type of analysis using privacy-preserving techniques or develop-ing systems that provide suitable transparency (such as audit trails) that we can balance against people’s reservations.
AcknowledgmentsThanks to Andrew Campbell (Dartmouth Col-
lege), Tanzeem Choudhury (Cornell Univer-
sity), Feng Zhao (Microsoft Research Asia),
and Lin Zhou (Shanghai Jiaotong University)
for their valuable feedback.
References1. N.D. Lane et al., “A Survey of Mobile
Phone Sensing,” Comm. Magazine, vol. 48,
no. 9, 2010, pp. 140–150.
2. J. Burke et al., “Participatory Sensing,”
Proc. Int’l Workshop World-Sensor-Web
(WSW 06), ACM Press, 2006, pp. 1–5.
3. E. Miluzzo et al., “CenceMe: Injecting
Sensing Presence into Social Networking
Applications,” Proc. 2nd European Conf.
Smart Sensing and Context (EuroSSC 07),
G. Kortuem et al., eds., Springer, 2007,
pp. 1–28.
4. T. Choudhury, Sensing and Modeling
Human Networks, PhD thesis, Massachu-
setts Inst. of Technology, Feb. 2004.
5. N. Eagle and A. Pentland, “Reality Min-
ing: Sensing Complex Social Systems,”
Personal Ubiquitous Computing, vol. 10,
no. 4, 2006, pp. 255–268.
6. J.-P. Onnela et al., “Structure and Tie
Strengths in Mobile Communication Net-
works,” Proc. Nat’l Academy of Sciences,
vol. 104, no. 18, 2007, pp. 7332–7336.
7. W. Mason, F. Conrey, and E. Smith,
“Situating Social Inf luence Processes:
Dynamic, Multidirectional Flows of
Influence within Social Networks,” Per-
sonality and Social Psychology Rev.,
vol. 11, no. 3, 2007, pp. 279–300.
8. M. McPherson, L. Smith-Lovin, and J.M.
Cook, “Birds of a Feather: Homophily in
Social Networks,” Ann. Rev. Sociology,
vol. 27, no. 1, 2001, pp. 415–444.
9. D. Kempe, J.M. Kleinberg, and E. Tardos,
“Influential Nodes in a Diffusion Model
for Social Networks,” Int’l Colloquium on
Automata, Languages and Programming,
vol. 3580, 2005, pp. 1127–1138.
10. N.D. Lane, Community-Guided Mobile
Phone Sensing Systems, PhD thesis, Dart-
mouth College, June 2011.
11. S. Consolvo et al., “Flowers or a Robot
Army? Encouraging Awareness and
Activity with Personal, Mobile Displays,”
Proc. 10th Int’l Conf. Ubiquitous Com-
puting (UbiComp 08), ACM Press, 2008,
pp. 54–63.
12. M. Anmol et al., “Social Sensing for Epi-
demiological Behavior Change,” Proc.
12th ACM Int’l Conf. Ubiquitous Com-
puting (Ubicomp 10), ACM Press, 2010,
pp. 291–300.
13. N.A. Christakis and J.H. Fowler, “The
Spread of Obesity in a Large Social Net-
work over 32 Years,” New England J. Med-
icine, vol. 357, no. 4, 2007, pp. 370–379.
14. J.N. Rosenquist, J.H. Fowler, and N.A.
Christakis, “Social Network Determi-
nants of Depression,” Molecular Psychia-
try, vol. 16, no. 3, 2010, pp. 273–281.
15. A. Krause et al., “Toward Community
Sensing,” Proc. 7th Int’l Conf. Information
Processing in Sensor Networks (IPSN 08),
IEEE CS Press, 2008, pp. 481–492.
16. L. von Ahn et al., “reCAPTCHA: Human-
Based Character Recognition via Web
Security Measures,” Science, vol. 321,
Aug. 2008, pp. 1465–1468.
17. T. Yan, V. Kumar, and D. Ganesan,
“CrowdSearch: Exploiting Crowds for
Accurate Real-Time Image Search on
Mobile Phones,” Proc. 8th Int’l Conf.
Mobile Systems, Applications, and Ser-
vices, ACM Press, 2010, pp. 77–90.
18. S. Reddy et al., “Examining Micro-
Payments for Participatory Sensing Data
Collections,” Proc. 12th Int’l Conf. Ubiq-
uitous Computing (Ubicomp 10), ACM
Press, 2010, pp. 33–36.
19. N.D. Lane et al., “Enabling Large-Scale
Human Activity Inference on Smart-
phones Using Community Similarity
Networks (CSN),” Proc. 13th Int’l Conf.
Ubiquitous Computing (Ubicomp 11),
ACM Press, 2011, pp. 355–364.
20. N.D. Lane et al., “Hapori: Context-Based
Local Search for Mobile Phones using
Community Behavioral Modeling and
Similarity,” Proc. 12th ACM Int’l Conf.
Ubiquitous Computing (Ubicomp 10),
ACM Press, 2010, pp. 109–118.
21. W. Pan, N. Aharony, and A. Pentland,
“Composite Social Network for Predict-
ing Mobile Apps Installation,” Proc. 25th
AAAI Conf. Artificial Intelligence (AAAI 11),
AAAI Press, 2011, pp. 821–827.
Nicholas D. Lane is a researcher at Microsoft
Research Asia. He’s an experimentalist
who likes to build prototype smartphone
sensing systems based on well-founded
computational models. Lane has a PhD
in computer science from Dartmouth
College. His dissertation pioneered com-
munity-guided techniques for learning
human behavior models better able to
cope with real-world conditions contain-
ing diverse user populations. Contact him
Selected CS articles and columns are also available for free at http://
ComputingNow.computer.org.
IC-16-03-Bwrs.indd 64 4/2/12 10:40 AM
