Real Time Geopolitical Risk Analysis

by

@aeneaswiener

About

• Aeneas Wiener

• Co-Founder of Cytora

• Background in Computational Physics / Electromagnetic Invisibility Cloaking

Geopolitical Risk

Tactical: As an airline, tell me every time there is a protest outside one of the airports

frequented by my fleet.

Strategic: What were some of the early warning signals for the Ukrainian

crisis of 2013?

“By 2020 more than 90% of what is knowable about the world will be in open source intelligence sources.”

Definitions• Geopolitical Risk Risk of loss faced by society and businesses as a result of actions taken by government or private actors such as protesters, terrorists and activists.

• Event ExtractionIdentification of real world events such as protests, suicide attacks and insurgency, which can be delineated in space and time.

Event Extraction• Problem statement

• Recognise 10+ event types from unstructured text

• Event information augmentation (geo, date)

• Event aggregation (group together mentions of same event)

• Two approaches

• Natural language processing (NLP) based pattern matching

• Machine learning (ML) based bag of words classifier

Date Type Lat Lon

04/11/14 Crme N 11° 5' 26'' E 11° 12' 56''

Event Extraction

User Interface

Stream Processing• Problem statement

• Billions of sentences per day in real time

• Historical archives (10+ years)

• Context

• Micro-services (web crawling, social media crawling, filtering, geo tagging, event extraction, entity extraction, translation, event persisting)

Apache Kafka• Characteristics

• Fault tolerant distributed message log

• Publish/subscribe semantics

• Can extent to queue and workers semantics

• Abstractions

• Topic

• Consumer

• Producer

Apache Kafka + Storm• Characteristics

• Version controlled processing graph (Storm topology)

• Horizontal scaling within Storm cluster (good resource utilisation)

• Streamparse library for integration with Python

• Issues

• Harder to debug than standalone micro-services

9 JuneIsis seize control of Mosul

1 JuneISIL 'kills 102-year-old man'

Event Correlation Analysis

Event Detection during 2013 Ukraine Crisis

The Cytora platform identified unrest in the East of Ukraine weeks before mainstream media and other providers.

Conclusions

• Events constitute a lens to observe the real world

• NLP / ML offer viable approaches for event extraction

• Apache Kafka + Storm enable event extraction at scale (It would take a human 5 years to analyse all the documents the Cytora system processes in a day.)

Senior Backend EngineerFull-time

Skills: Python Distributed Systems Machine Learning

Full-stack EngineerFull-time

Skills: Python JavaScript HTML/CSS

Jack Palfrey

Location: 113 Shoreditch High Street

Dev-ops EngineerFull-time

Skills: Fabric, Asible or Chef PostgreSQL/ElasticSearch Monitoring / log shipping UNIX administration

Forward DeployedEngineerFull-time

Skills: Python Project management