Embed Size (px)
Citation preview
YAB !(Yet Another Build tool)
Christos KK Loverdos
[email protected]@loverdos
Athens Scala Meetup, 2015-05-05
AgendaBuild systems/tools
Motivation
Key features
Design & semantics
Examples
Roadmap (-ish)
Build systems/toolsmake
ant
maven
sbt
gradle, pants, buck, bazel, …
Makeworld: cd src; $(MAKE) install
clean: cd src; $(MAKE) clean rm -f *~
test10: tests/test10.ott bin/ott \ -o out.thy -o out.v …
Make$ make world …
$ make clean …
$ make test10 …
Ant<project name=“foo” default=“compile”> <target name=“compile” depends=“init”> <mkdir dir=“${build.dir}”/> <copy todir=“${build.dir}”> <fileset dir=“${basedir}”> <exclude name=“**/VERSION”> </fileset> </copy> </target> </project>
Ant$ ant …
$ ant init …
$ ant compile …
Maven<dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency>
Maven<plugin> <groupId>net.alchim31.maven</groupId> <artifactId>scala-maven-plugin</artifactId> <version>3.2.0</version>
<configuration> <scalaVersion>2.11.6</scalaVersion> </configuration> </plugin>
Maven$ mvn clean compile package …
$ mvn dependency:resolve …
$ mvn clean deploy …
SBT (Simple Build Tool)
“sbt is a neat tool and I have been using it almost exclusively lately” — me, 2008-12-05, simple-build-tool group
Contributed “sbt.bat” via email :-)I was on Windows back then :-)
SBT (Simple Build Tool)
De-facto (?) Scala build tool
Turing-complete DSL (Full Scala)
Plugins offer extra functionality
Not-so-simple after all …
Motivation
I had an itch to scratch
Christos KK [email protected]/loverdos
ckkloverdos.com
Building without the closed-world assumptionIntegrating enterprise build artifacts using Scala
ScalaDays
2010
Motivation
Kinda frustrated with maven
Motivation
Although used SBT professionally, it’s “uncontrolled”/“undisciplined” turing completeness ultimately puzzled me
Motivation
I though that the Problem Domain of a build tool did not get the attention it deserved and there was a clean room for investigation
Motivation
I needed a side project for my own reasons
Key features
Clean separation of Data & Behaviour
Objects are not the answer to everything, always
Key featuresData-first approach
Your build is defined by data
Dependencies, what to execute etc are data ==> purely functional
Execution of data-defined orchestration is via precise behaviours; think services.
Key features
Immutability all the way
Key features
Direct Acyclic Graph
Coarse-grained cycle detection ==> better software engineering practices
Cycles mean no build at all
Key features
Do not (re)invent any wheel
Maven Central is an achievement
Re-use existing libs via maven coordinates
Key features
Make it as simple as possible but no simpler
Key features
Respect the developer
There is only one way to do this:
Treat them as end-users, not hackers
Key featuresMulti-paradigm
One repo per project
One repo for all (Google-style)
Hybrid projects
mount external dependencies (à la FS)
Key features
Hash-based change detection
Reproducible builds
Keeps track of everything that is used in the build process
Design & SemanticsRecipes define what we can make and how
A JAR, a WAR
the output of javac, scalac etc
firing up a server (!) (think: jetty)
Design & Semantics
Ingredients are what the recipes use
Each recipe defines its ingredients
Some are required; others not
Design & SemanticsOrders are what we give to YAB
“order a scala compilation”
An order refers to the execution of a recipe
We provide it with values for the needed ingredients
Design & SemanticsAnd that’s it:
Ingredients (Data)
Recipes (Behaviour)
Orders (Data-first recipe orchestration)
Design & SemanticsBuild Components or just Components
Hierarchical project structure
think: sub-projects, maven modules etc
Component <==> yab.conf
yab-vfs/yab.conforders { compile { type: "scala/scalac" ingredients: [ { scalaVersion: "2.11.1" } { Order: "/remotes/mlpipe::compile" } { Order: "/remotes/mlpipe-collections::compile" } ] } jar { type: "jar/make" ingredients: [ { Order: compile } ] } default=jar }
Command-line$ yab /yab-vfs::compile
[yab] […] /remotes/mlpipe::compile [ /yab-vfs::compile ] [yab] [✓] /remotes/mlpipe::compile [0.545 sec] … [yab] […] /yab-vfs::compile [yab] [✓] /yab-vfs::compile [0.047 sec] [0.761 sec in total] [yab] ⤑ FileRoot(=.yab/components/yab-vfs/product/scalac) [yab] Finished in 2.700 sec
yab-spec/yab.conforders { compile { type: "scala/scalac" ingredients: [ { scalacOptions: [ "-deprecation", "-unchecked", "-feature" "-optimise" "-language:existentials,higherKinds,implicitConversions" ] } { Order: “/remotes/mlpipe::compile" } { MavenArtifact: “com.typesafe:config:1.2.1" } ] } }
Roadmap
Write docs
Finish latest iteration (some service-orientation love, cleanup)
Open source
Facts/ImpressionsIt has been fun designing & developing YAB so far.
From immutable ASTs to clean separation of Data & Behaviour, the challenges are constant and rewarding.
The core principles can go beyond the build process.
Thank you
