The Future of Financial Information Services
QCON Shanghai Nov 2013
Amish Gandhiwww.perpetualny.com
Amish GandhiFounder and Principal at Perpetual: Product innovation and development for financial, media and telecom www.perpetualny.com
Financial services background• Worked at Thomson Reuters for 3 years
– Global Product Manger for Reuters Messenger – Product manager for financial video service Reuters Insider
MS Computer Science from Univ of Texas, Austin
BS Computer Science from Bombay University
Finance
Media
Telecom
The Future of Financial Information Services
• The rise of consumer internet– 1995– 2007– 2013
• How did we get here?– Resulting technologies
• Difference between Consumer and Finance– Innovators Dilemma in Finance
• 10 Predictions for the Future of Finance
Outline
The Future of Financial Information Services
Financial Service Providers
Rise of consumer tech
Consumer Internet/Tech Landscape Pre 1995 Financial Technology Landscape Pre 1995
ServiceDistribution
Motorola Pageboy
Mainframe
1995
2007
Gaming-inspired system that would track and display the expertise of
users around the world.
Ideo Bloomberg Concept
Electronic notepad reused like a virtual Post-it
Bloomberg Wherever—allows users to take a small-scale Bloomberg terminal device beyond their desks.
Left to right info flow
UI similar to modern websites
2007
2013
Realtime
How did we get here?
SearchSocialCommunicationEcommerceGamingUser InterfaceInternet of things
??
?
?
Before 1995
Search
??
?
?
1995 Onwards
Search
Search
-Snail mail-/dev/null
Before 1995
Social
Comments
Social
Social
Big Data Ecosystem
Search/Social
WhatsApp Skype BBM
3G 4G
Communication
1990’s and pre
$$
$
$
ECommerce
Massively multiplayer online role-playing game (MMORPG) & Social Gaming
Gaming
Radian 6Consumer Products: Professional User Interfaces
User Interface
InternetInternet
HTML5CSS3Firefox/ChromeJavascriptJS frameworksBackboneEmberAngular
INTERNET OF THINGSSensorsEvent driven computingWebsockets vs. http Bluetooth and other NFCMaterial scienceElectromyography-sensorsAccelerometersSpeech recognitionPower consumption…
GAMINGAdvanced graphicsGame theory implementedAdvanced GPUsAdvanced GPU programming
UI:PROCESSUCDA/B testingLeanUser testing
ECOMMERCECloud Computing (EC2)Online paymentsMobile paymentsOnline payment gatewaysBitcoin…
COMMUNICATIONSmartphonesAndroidiOS 3G/4GSMSMMSiMessageErlangWebRTC… SOCIAL
NoSQL CassanrdaHbase/HadoopTwitter StormLocation based servicesNatural Language Processing…
SEARCHMapReduceGoogle BigtableGoogle BigQueryLAMP/Commodity scalingLuceneSOLRElastic SearchAjaxTwitter StormApache S24…
Technologies Born from Consumer
Technologies born from Consumer
USER INTERFACE
Pre 1995
AWS Android Mapreduce Bigtable Hadoop Hbase NodeJS Scala Cloud Computing Big Data D3.js Coffee Script/Backbone Data Visualization Elastic search Nginx Hudson GO R require.js less/sass/compass, HTML5…
Present
Consumer Internet/Tech Landscape
Difference between Consumer and Finance Applications
Consumer Experiences
FinancialExperiences
Highly competitive high volume market driven
evolution
Established traditionalbusiness model
ScalableFast
Reliable
SecureCompliant
OpenConnected
Rich
Innovators Dilemma
HTML5CSS3Firefox/ChromeJavascriptJS frameworksBackboneEmberAngular
INTERNET OF THINGSSensorsEvent driven computingWebsockets vs. http Bluetooth and other NFCMaterial scienceElectromyography-sensorsAccelerometersSpeech recognitionPower consumption…
GAMINGAdvanced graphicsGame theory implementedAdvanced GPUsAdvanced GPU programming
UI:PROCESSUCDA/B testingLeanUser testing
ECOMMERCECloud Computing (EC2)Online paymentsMobile paymentsOnline payment gatewaysBitcoin…
COMMUNICATIONSmartphonesAndroidiOS 3G/4GSMSMMSiMessageErlangWebRTC… SOCIAL
NoSQL CassanrdaHbase/HadoopTwitter StormLocation based servicesNatural Language Processing…
SEARCHMapReduceGoogle BigtableGoogle BigQueryLAMP/Commodity scalingLuceneSOLRElastic SearchAjaxTwitter StormApache S24…
Technologies Born from Consumer
Technologies born from Consumer
USER INTERFACE
Innovators Dilemma
AWS Android Mapreduce Bigtable Hadoop Hbase NodeJS Scala Cloud Computing Big Data D3.js Coffee Script/Backbone Data Visualization Elastic search Nginx Hudson GO R require.js less/sass/compass, HTML5…
The Future of Financial Information Services
10 Predictions
There will be a revolution in financial information visualization
Prediction 1
Visualization
-Financial Data is complex & large scale-Visualization of financial mostly same last 15 years-Current visualization relies on a variety of charts-Consumer BigData visualization has evolved rapidly
Products at the forefront Current Custom HTML5 Support/
Javascript API
Prediction 1
Prediction 1
Technologies in play
Emerging Technology: Present Future Technology >3 years:Gaming Engines
-Canvas -Raphael-Protovis
Prediction 1
Cloud services will be used much more for computation
Prediction 2
GPUs in Finance
• Significant advancement in GPUs for gaming physics engines• High speed computation• Can have 512 cores (<10 in avg. CPU)• Can have 10x bandwidth• 100 X fast calculation of typical derivative model
Pros: Ideal for well defined structured computationCons: Not very flexible, steep learning curve
Prediction 2
GPU Opportunities in Finance
• Derivatives Pricing• Speeding up risk analysis (can take days/weeks)• Trading strategy prospecting (back testing)• HFT (complex event processing)• Tick data (added value data feeds in real time)• Data visualization
High Performance Computing (HPC) to leverage GPU arrays available on Amazon
Prediction 2
Realtime solutions will be adapted from consumer world
Prediction 3
Realtime in Consumer World
• Ad serving user real-time • Capture and process model
Advertising Social media data processing• Multiple streaming sources• Capture and process model• Real-time ad response• Real-time engagementAnalytics
• Usage in realtime• Content usage tracking to
adapt strategies• Usage tracking to manage
load
Messaging and Communication• 2 way real-time comms
– Mobile vs - Text– Desktop -Images
-Video
Prediction 3
Realtime in finance
Use-cases– Exchange streaming pricing– Reference data access and
display– Liquidity analysis & risk– News– Communication– Alerts– Active compliance…social media
Technical solutions today:-MQ Series/Tibco-FPGA processors-Custom C++ solutions-Traditional RDBMS-Rules engines-Processing-Storage…stream processing
Capture Stream Processing
Resultstorage
Distribution
Price servers
Marketmovement
News
Rates changes
Realtime Data Processing Model
Ref: Real Time Data Processing - Technical Terrain by Prakash Khemani
Prediction 3
Capture
Stream Processing
Resultstorage
Distribution
Exchange price servers ticker plants
Aggregates and indices
News feeds
Rates and otheradjunct prices
Realtime modeling
Best-effort delivery guarantee
-Write-ahead-logging -At-least-once delivery guarantees
Persistent queue
-Price queue-Index queue-News queue
Millwheel-Large scale fault tolerant stream processing-Used for Google Zeitgeist
JMS
-Transactional guarantees-Strict ordering guarantees-Limited throughput
In-memory database
Sensors
Realtime Data Processing Model
Prediction 3
Capture
Twitter Storm: at-least once processing guarantee for every record
Resultstorage
Distribution
Exchange price servers ticker plants
Aggregates and indices
News feeds
Rates and otheradjunct prices
Persistent Queue Input
YARN Hadoop NextGen MapReduce
Compute node/cluster Management
-Pull data from persistent Q-Scheduling computation -Maintaining checkpoints
Zookeper
Trident: Exactly-once processing mode of storm. Allows you to seamlessly mix -high throughput + stateful stream processing-low latency distributed querying
Stream Processing
Streaming
Local state
-Does not persist output at every stage-Keeps track of each record's lineage -Can replay the data to recreate the records when needed-Fast-Cons: Unordered batch completions, low guarantees
PregelAll in the same system:-Event Capture Storage-Computation State-Result Storage
(Built on Bigtable)
Realtime modeling
Sensors
Prediction 3
Capture
Result Storage Distribution
Exchange price servers ticker plants
Aggregates and indices
News feeds
Rates and otheradjunct prices
Persistent Queue Input
Stream Processing
-Store batches into storage-Do the aggregations at read time
-Buffer up the updates to many keys-Do a sequential merge with historical data
Distributed, scalable Time Series Database (TSDB) written on top of HBase
Specializes in exporting key/value data from Hadoop
Realtime modeling
Sensors
Realtime Data Processing Model
Prediction 3
CaptureResult Storage Distribution
Exchange price servers ticker plants
Aggregates and indices
News feeds
Rates and otheradjunct prices
Persistent Queue Input
Stream Processing
Realtime modeling
Sensors
Prediction 3
Wave of improved general UX
Prediction 4
Mozilla labs design contest: Flickr @peterheads
Prediction 4
Prediction 4
Prediction 4
Prediction 4
Prediction 4
Prediction 4
Prediction 4
Prediction 4
3
4
2
1
1
1 5
Prediction 4
There will be a major change in the tiered architecture with HTML5
Prediction 5
HTML 5
Offline DatabaseCross BrowserMultimediaMobileOffline App supportGeo-locationImproved forms support
Prediction 5
Current
Windows/HTML
Front End
MiddleLayer
(App server with business and
transformation logic)
Pricing Engine
NewsService
Research
Historical Data
Events
Other…
Prediction 5
Future
HTML 5UI
Pricing Engine
NewsService
Research
Historical Data
Events
Other…
SOA (Service Oriented
Architecture)
BA
External 3rd party services
Prediction 5
Technologies in-play
• Websockets– Realtime bi-directional– Event programming : no http overhead– Javascript coming to the forefront on top of web
sockets• Server support for Websockets
Products
Prediction 5
In-desktop communication will be enhanced with Web RTC and similar technology
Prediction 6
Communication over financial products
• Skype/Softphones exploded on the web• PSTN/VoIP/Cellular phones still used in finance
environments • Communication endpoints will emerge on
desktops & mobile finance apps – Enabling realtime comm. including video
conferencing
Prediction 6
Technologies• Jabber / XMPP– Open syndication of messaging across networks
• Web RTC will play a big role• Enhanced on-desktop communication and FIX
integration• Rise of Erlang for messaging eg eJabberd
Communication over financial products
Prediction 6
The rise of Erlang
• Large scale concurrency• Soft real-time• Distributed • Hardware interaction• Software maintenance on-the-fly• Fault tolerance …. and many more
Prediction 6
Analyst research will take on a much richer format beyond PDF and will be published to
permissioned microsites
*Same for other frequently shared information eg earnings calls
Prediction 7
Analyst Research
• Current analyst research reports– Static PDF
Physical file sent through several intermediaries
Advanced entitlement model restricting access.
Distribution through financial research aggregators
Prediction 7
Relevant Technologies
• HTML5• CSS3• Web app stack: PHP/Python/Ruby• JS frameworks– Backbone– Ember– Angular......
Prediction 7
Financial training will be revolutionized to incorporate gaming and other aspects seen on
consumer web
Prediction 8
Financial Training
There is a vast amount of financial related trainingSeries 7Series 86/87 certifications CFA prepCompliance related training
• Education based apps and readers are much more widely available on mobile
• Financial training will see a rapid adoption of mobile and gaming based innovations
Prediction 8
Technologies
• Game theory• iOS/Android/Windows Phone• ePub adoption
Prediction 8
Android based OS will become a dominant force, starting with consumer banking
Prediction 9
Android Usage
More than 1 million new Android devices are activated worldwide. Daily.
1.5 billion downloads a month and growing.
The next 1 billion internet activations in remote areas will be on an Android device.
Prediction 9
Technology
• Android• vs. Chrome OS
Prediction 9
Direct sensor tracking for assets like Commodities & Energy
Prediction 10
Changes• From vessel to batch level sensors
– Deeper insight into supply chain
• Connected sensors tracing – Volume– Quality– Speed– Location
• Combined with following real-time data– Weather – Transportation – Government events
Access to high volume, low latency data for new level of accuracy
Prediction 10
Predictions Summary
1. Revolution in financial information visualization2. Much more GPU and GPU cloud computing3. Realtime solutions will be adapted from consumer world4. Wave of improved general UX on financial terminals5. Major change in the tiered architecture with HTML5 6. Communication will be enhanced with Web RTC & similar7. Analyst research go from static PDFs to dynamic microsites8. Financial training will incorporate social and gaming aspects9. Android based OS will dominate for financial info services10. Direct sensor tracking for assets like commodities & energy
www.perpetualny.com