Crowdsourcing Approaches for Smart City Open Data Management

Crowdsourcing Approaches for Smart City Open Data Management

Edward Curry & Adegboyega Ojo

Insight @ NUI Galwayed.curry@insight-centre.orgwww.edwardcurry.org

About Me

• Researcher in both Computer Science and Information Systems

• Green and Sustainable IT Research Group Leader in DERI/Insight NUI Galway

Some Background

Multi-year research on state of research and practice of smart cities to inform Next Generation Smart City Design and Policy

Part of an International Smart Cities Research/Practice Consortium composed of international research teams from the US, Canada, Mexico, Colombia, China and Ireland.

Designing Next Generation Smart City Initiatives - SCID

Ojo, A., Curry, E., and Janowski, T. 2014. “Designing Next Generation Smart City Initiatives - Harnessing Findings And Lessons From A Study Of Ten Smart City Programs,” in 22nd European Conference on Information Systems (ECIS 2014)

Open Data as a Smart City Imitative

Ojo, A., Curry, E., and Sanaz-Ahmadi, F. 2015. “A Tale of Open Data Innovations in Five Smart Cities,” in 48th Annual Hawaii International Conference on System Sciences (HICSS-48)

Open Data Powering Smart Cities

Economy

Energy Environment

Education

Health & Wellbeing

Tourism Mobility Grovenance

An Open Innovation Economy

Initial findings of the study are consistent and support the notion of an open data oriented smart city as an:

“Open Innovation Economy”

We are now investigating Crowdsourcing as a means of increasing Citizen engagement and participation within a smart city’s open innovation ecosystem

Introduction to Crowdsourcing

Coordinating a crowd (a large group of workers)to do micro-work (small tasks) that solves problems (that computers or a single user can’t)

A collection of mechanisms and associated methodologies for scaling and directingcrowd activities to achieve goals

Related Areas Collective Intelligence Social Computing Human Computation Data Mining

A. J. Quinn and B. B. Bederson, “Human computation: a survey and taxonomy of a growing field,” in Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, 2011, pp. 1403–1412.

9

Crowdsourcing Landscape

When Computers Were Human

Maskelyne 1760Used human computers

to created almanac of moon positions

– Used for shipping/navigation

Quality assurance– Do calculations twice– Compare to third verifier

D. A. Grier, When Computers Were Human, vol. 13. Princeton University Press, 2005.

When Computers Were Human

Audio Tagging - Tag a Tune

Image Tagging - Peekaboom

Protein Folding - Fold.it/

ReCaptcha

OCR ~ 1% error rate 20%-30% for 18th and

19th century books 40 million ReCAPTCHAs

every day” (2008) Fixing 40,000 books a

day

Enterprise Examples

Understanding customer sentiment for launch of new product around the world.

Implemented 24/7 sentiment analysis system with workers from around the world.

90% accuracy in 95% on content

Categorize millions of products on eBay’s catalog with accurate and complete attributes

Combine the crowd with machine learning to create an affordable and flexible catalog quality system

Spatial Crowdsourcing

Spatial Crowdsoucring requires a person to travel to a location to preform a spatial task Helps non-local requesters through workers in targeted

spatial locality Used for data collection, package routing, citizen

actuation Usually based on mobile applications Closely related to social sensing, participatory sensing,

etc. Early example Ardavark social search

Sensing

Credits: Albany Associates, stuartpilrow, Mike_n (Flickr)

Computation Actuation

Human Powered Smart Cities

Leverages human capabilities in conjunction with machine capabilities for optimizing processes in the cyber-

physical-social environments

Citizen Sensors

“…humans as citizens on the ubiquitous Web, acting as sensors and sharing their observations and views…”

Sheth, A. (2009). Citizen sensing, social signals, and enriching human experience. Internet Computing, IEEE, 13(4), 87-92.

Air Pollution

Crisis Response

Citizens as Sensors

23

Haklay, M., 2013, Citizen Science and Volunteered Geographic Information – overview and typology of participation in Sui, D.Z., Elwood, S. and M.F. Goodchild (eds.), 2013. Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI) in Theory and Practice . Berlin: Springer.

HumanVisual perceptionVisuospatial thinkingAudiolinguistic abilitySociocultural

awarenessCreativityDomain knowledge

MachineLarge-scale data

manipulationCollecting and storing

large amounts of dataEfficient data

movementBias-free analysis

Human vs Machine Affordances

R. J. Crouser and R. Chang, “An affordance-based framework for human computation and human-computer collaboration,” IEEE Trans. Vis. Comput. Graph., vol. 18, pp. 2859–2868, 2012.

Generic Architecture

Workers

Platform/Marketplace (Publish Task, Task Management)

Submi

t Task Colle

ct

Answe

r

Find

Task

Retur

n

Answe

r

Requestors

1.

2.

4.

3.

Platforms and Marketplaces

Core Design Questions

GoalWhat

Why

IncentivesWho

Workers

HowProcessMalone, T. W., Laubacher, R., & Dellarocas, C. N.

Harnessing crowds: Mapping the genome of collective intelligence. MIT Sloan Research Paper 4732-09, (2009).

Setting up a Crowdsourcing Process1 – Who is doing it?

Hierarchy (Assignment), Crowd (Choice)

2 – Why are they doing it? Money ($$££), Glory (reputation/prestige), Love (altruism,

socialize, enjoyment), Unintended by-product (e.g. re-Captcha, captured in workflow), Self-serving resources (e.g. Wikipedia, product/customer data), Part of their job description

Determine pay and time for each task Marketplace: Delicate balance (Money does not improve quality but can

increase participation) Internal Hierarchy: Engineering opportunities for recognition: Performance

review, prizes for top contributors, badges, leaderboards, etc.

3 – What is being done? Creation Tasks: Create/Generate/Find/Improve/ Edit / Fix Decision (Vote) Tasks: Accept/Reject, Thumbs up / Down,

Vote

4 – How is it being done? Identify the workflow: Integrate in workflow (“rating”

algorithm) Identify the platform (Internal/Community/Public) Identify the Algorithm (Data quality, Image recognition,

etc.)

Summary

29

Analytics & Algorithms

Entity LinkingData Fusion

Relation Extraction

Human Computation

Relevance JudgmentData VerificationDisambiguation

Better Data

Internal Community - Domain Knowledge - High Quality Responses - Trustable

Web Data

Databases

Sensor Data

Programmers Managers

External Crowd - High Availability - Large Scale - Expertise Variety

References & Further Information

Ojo, A., Curry, E., and Janowski, T. 2014. “Designing Next Generation Smart City Initiatives - Harnessing Findings And Lessons From A Study Of Ten Smart City Programs,” in 22nd European Conference on Information Systems (ECIS 2014)

Ojo, A., Curry, E., and Sanaz-Ahmadi, F. 2015. “A Tale of Open Data Innovations in Five Smart Cities,” in 48th Annual Hawaii International Conference on System Sciences (HICSS-48)

Curry, E., Freitas, A., and O’Riáin, S. 2010. “The Role of Community-Driven Data Curation for Enterprises,” in Linking Enterprise Data, D. Wood (ed.), Boston, MA: Springer US, pp. 25–47.