SmartNews's journey into microservices

Takumi Sakamoto Livesense TechNight

2015.09.29

Takumi Sakamoto @takus

News Delivery Pipelines

36 months ago The first system is developed

Crawler Analyzer Indexer

On memory DB API

Monolithic App

Social Media

Article Lists

12 months ago Split crawlers

Crawler

Queue

Monolithic App

Analyzer Indexer

On memory DB API

https://medium.com/@takus/velocity-conference-2015-wrap-up-velocityconf-f3811e05f1e8

4 months ago At Velocity Conference

2 months ago Dive into the microservices world

Crawler

Kinesis

IndexerAnalyzer

Kinesis CloudSearch

Why microservices?

• A lot of different types of services are required for us

• crawler, image analyzer, content extractor ...

• Adapt to technical progress / market change

• every system should be replaceable

• Make developers happy :)

• avoid to maintain "legacy" "large" "monolithic"

• enable us to develop any languages

A part of microservices

Source A

Analyzer Indexer Search API A

PhotoqualCaffe

Reporting

API Gateway

Realtime feedback

Image Analysis

Extractor

Content extraction

Source B

Search API B

Ranking algorithm A/B test

Language detecter

Crawling

Language detection

Spark Streaming

KPI Dashboard

Admin

Operation

References (Japanese only)

Stream 処理と Offline 処理の統合

http://bit.ly/1VnQQao

スマートニュースの世界進出を支えるログ解析基盤

http://bit.ly/1MCOyBX

SmartNews の Webmining を支えるプラットフォーム

http://bit.ly/1RcVGqk

Journey into microservices

When you use microservices you have to work on automated deployment, monitoring, dealing with failure, eventual consistency, and other factors that a distributed system introduces.

http://martinfowler.com/bliki/MicroservicePremium.html

Automated Deployment with AWS CodeDeploy

Today's topic

Deployment is not easy

• Manage complexity

• multiple roles

• multiple environments

• auto scaling

• Avoid downtime

• JVM based applications doesn't support hot deploy

• Monitoring

• stop deployment if something wrong

• which instances are succeeded or failed

AWS CodeDeploy

http://www.slideshare.net/AmazonWebServices/aws-codedeploy-manage-deployment-complexity

2W1H configurations

http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html

Where?How?What?

Deployment Flow

http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html

What?

How?

Where?

What to Deploy?

• Prepare your "revision" archive

• appspec.yml

• application (WAR/JAR etc...)

• (optional) scripts

• register to ELB / deregister from ELB

• pre-compile assets

• waiting cache warmup

App Spec

How to Deploy?

Where to Deploy?

EC2 Tag / Auto Scaling Group

My favorites

• Simple, Fast, Flexible, Free

• Require small changes for current system

• Integration w/ AWS Auto Scaling

• deploy last successful revision to a new instance

• wait for registering ELB until finishing deployment

Pitfall

Exclusive lock is required :P

Deployment is skipped during scale-out

Automated Deployment at SmartNews

Deployment pipelines

Slack notification

Role / Env / Version Deployment status

Commits

Datadog monitoring

Datadog alerts

Manage alerts by DSL

Most of alerts are created automatically Powered by airbnb/interferon

Docker for C++ apps

Towards future...

We’re hiring!!! Join our journey into microservices

Site reliability engineer Productivity engineer Web application engineer iOS/Android engineer ML/NLP engineer Ad engineer

http://about.smartnews.com/en/careers/

Wait for you @ SmartKitchen