Embed Size (px)
DESCRIPTION
How to build Spray How to build Akka How to Monitor them
Citation preview
Scala {Culture}
Who Am I• Now: Dynalyst • Lang: Korean(30+ years) Japanese(8+ years) JAVA(8.2+ years) Scala(a year)
• AdTech: AMoAd DSP CAA Reward
What Dynalyst• Dynamic Retargeting + Analyst
•ユーザーの趣味嗜好に適した広告配信を行う国内初のスマートフォンに特化したダイナミックリターゲティング広告
Scala USAGE
Today’s Talk
•How to build Spray
•How to build Akka
•How to Monitor them
mark ad bid imp click cv
redshift
storedimportingloggingrequest tracking
Dcontinuous importing
redshift
reporting & bidding optimizationsummaringstored
date imp click cv
2014/09 10,000 500 10
2014/08 15,000 700 25
continuous reporting D
Benchmark
What is Spray?
spray is an open-source toolkit for building REST/HTTP-based integration layers on top of Scala and Akka. Being asynchronous, actor-based, fast, lightweight, modular and testable it's a great way to connect your Scala applications to the world.
Concurrency
Future[T]
implicit val ec: ExecutionContext
val f = Future { “hello world” }
// Future[String]
Future[T]
implicit val ec: ExecutionContext
for { greeting ← Future(“hello world”) name ← Future(“Han”)} yield s”$greeting, $name”// Future[String]
FutureDirectives
• spray-routing directives
• non-blocking actor thread
FutureDirectives
def onComplete[T] (future: ⇒ Future[T]) (implicit ec: ExecutionContext) :Directive1[Try[T]]
implicit val ec: ExecutionContext
def respondWithBidding = { onComplete(bidding) { case Success(bid: Bid) ⇒ … case Success(noBid: NoBid) ⇒ … case Failure(e) ⇒ … }}
FutureDirectives
Type of bidding
def bidding:Future[A<:BidResponse]
Implementation
def bidding:Future[A<:BidResponse]
!
• simple but bad implementation
Future { // All bidding computations :(}
Application Layers
Spray Router Handle raw level http request & response
Bidding Business Logic
External Resources"Fetch bidding candidates from data stores
Global Dispatcher
Application Layers
Spray Router Handle raw level http request & response
Bidding Business Logic
External Resources"Fetch bidding candidates from data stores
Application Layers
Spray Router Handle raw level http request & response
Bidding Business Logic
External Resources"Fetch bidding candidates from data stores
Router Dispatcher
Service Dispatcher
Repository Dispatcher
Implementationtrait Router { implicit val ec = RouterDispatcher def handle: Future[BidResponse]}
trait Service { implicit val ec = ServiceDispatcher def bidding: Future[Bidder]}
trait Repository { implicit val ec = RepositoryDispatcher def candidates: Future[List[Candidate]]}
How to test those Future[T] things
• Specs2 - Matcher[Future[T]]
bidding must beEqual(expected).await
bidding must beEqual(expected).await( retries = 2, timeout = 100.millis)
Abstract Future
http://logji.blogspot.jp/2014/02/the-abstract-future.html
Abstract Futuretrait Router[M[_]] { implicit val M: Monad[M] def handle: M[BidResponse]} trait Service[M[_]] { implicit val M: Monad[M] def bidding: M[Bidder] = M.point(bidder) def bidder = …} trait Repository[M[_]] { implicit val M: Monad[M] def candidates: M[List[Candidate]]}
Type class & Implicit
object Service { implicit def fs = new Service[Future] { implicit val ec = ServiceDispatcher val M = Monad[Future] } implicit def ids = new Service[Id] { val M = Monad[Id] }} // type Id[+A] = A
Test Again
val s = Service.ids
s.bidding must beEqual(expected)
M[_] for Monad
• Monad is a structure that represents computations defined as sequences of steps
• Don’t dive into “Monad” here!
redshift
storedimportingloggingrequest tracking
Dcontinuous importing
mark ad bid imp click cv
redshift
reporting & bidding optimizationsummaringstored
date imp click cv
2014/09 10,000 500 10
2014/08 15,000 700 25
continuous reporting D
Master/Worker
http://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2
Overview
Overview
How to Monitor Akka
• Typesafe Console
http://resources.typesafe.com/docs/console/manual/overview.html
• Kamon
http://kamon.io/
Typesafe console• Great user interface
• Support by Typesafe
• Expensive
Kamon• Open Source - Free
• Metrics: Akka, Spray, Play(over 50 categories)
• StatsD, New Relic, Datadog
• Migration could be hard
Kamon meets Zabbix
Running Thread Count
Summary
• Future Directive on Spray
• Abstract Future(Type class & Implicit)
• Akka master/worker pattern
• Monitoring Akka with Kamon
Best Practices?
• https://github.com/alexandru/scala-best-practices
• So many “MUST NOT, SHOULD NOT”
• In my opinion, Trial and Error to find right patterns
