Using Language Oriented

Programming to Execute

Computations on the GPUComputations on the GPU

Robert Pickering, ALTI

About the Presenter• Using F# for about 6 years

• Oldest F# user outside of Microsoft

• Written a book about F#

(now in its second edition)

• Spend at least 2 years as a professional

functional programmer

2

Contact me:

robert@strangelights.com

http://strangelights.com/blog

functional programmer

• I have 3 cats

ALTI

• Large French services company

• Consulting, engineering and

training

• Believe in helping clients • Believe in helping clients

discover upcoming niches

• Already offer F# training and

consulting

What is Language Oriented

Programming ?

• Creating programs that are an abstract

description of the a problem

• This description can then either be • This description can then either be

interpreted, compiled, or analyzed in another

way

LOP & Combinators

• F# has two main approaches to Language

Oriented Programming:

– Reinterpreting the F# syntax though the

quotations systemquotations system

– Using combinators create a new syntax

… both approaches are very similar

What is a Combinator?

A combinator is a higher-order function that

uses only function application and earlier

defined combinators to define a result from its defined combinators to define a result from its

arguments.

Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic

Combinatory Logic in Computing

In computer science, combinatory logic is used

as a simplified model of computation, used in

computability theory and proof theory. Despite

its simplicity, combinatory logic captures many its simplicity, combinatory logic captures many

essential features of computation.

Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic

Combinator Library

"A combinator library offers functions (the

combinators) that combine functions together to make

bigger functions"[1]. These kinds of libraries are

particularly useful for allowing domain-specific

programming languages to be easily embedded into a programming languages to be easily embedded into a

general purpose language by defining a few primitive

functions for the given domain.

Souce: Wikipedia http://en.wikipedia.org/wiki/Combinator_library

[1] “A History of Haskell” Hudak, Hughes, Peyton Jones, Wadler

History of Haskell: Combinator Libraries

What is a combinator library? The reader will

search in vain for a definition of this heavily

used term, but the key idea is this: a combinator

library offers functions (the combinators) that library offers functions (the combinators) that

combine functions together to make bigger

functions.

History of Haskell: Combinator Libraries

What is a combinator library? The reader will

search in vain for a definition of this heavily

used term, but the key idea is this: a combinator

library offers functions (the combinators) that library offers functions (the combinators) that

combine functions together to make bigger

functions.

History of Haskell: Combinator Libraries

Another productive way to think of a

combinator library is as a domain-specific

language (DSL) for describing values of a

particular type.particular type.

What is a Domain Specific Language?

A programming language tailored for a particular application

domain, which captures precisely the semantics of the

application domain -- no more, no less.

A DSL allows one to develop software for a particular A DSL allows one to develop software for a particular

application domain quickly, and effectively, yielding

programs that are easy to understand, reason about, and

maintain.

Hudak

Combinators vs DSLs

• Combinartor libraries are a special case of DSLs

– Sometimes called DSELs (Domain Specific Embed languages)

• DSELs have several advantages:

― Inherit non-domain-specific parts of the design.― Inherit non-domain-specific parts of the design.

― Inherit compilers and tools.

― Uniform “look and feel” across many DSLs

― DSLs integrated with full programming language, and with each other.

• DSELs one disadvantage:

― Constrained by host language syntax and type system

What Makes F# a Suitable for DSLs ?

• Union Types / Active Patterns

– type Option<'a> = Some x | None

• Lambda functions

– fun x -> x + 1– fun x -> x + 1

• Define and redefine operators

– let (++) x = x + 1

• Define custom numeric literals

– let x : Expression = 1.0N

Union Types – The Option Type

// The pre-defined option type

type Option<'a> =

| Some of 'a

| None

// constructing options// constructing options

let someValue = Some 1

let noValue = None

// pattern matching over optionslet convert value =

match value with| Some x -> Printf.sprintf "Value: %i" x| None -> "No value"

Union Types - Trees

// a binary tree definition

type BinaryTree<'a> =

| Node of BinaryTree<'a> * BinaryTree<'a>

| Leaf of 'a

// walk the tree collection valueslet rec collectValues acc tree =

match tree with| Node(ltree, rtree) ->

// recursively walk the left treelet acc = collectValues acc ltree// recursively walk the right treecollectValues acc rtree

| Leaf value -> value :: acc// add value to accumulator

Using the Tree

// define a tree

let tree =

Node(

Node(Leaf 1, Leaf 2), Node(Leaf 1, Leaf 2),

Node(Leaf 3, Leaf 4))

// recover all values from the leaves

let values = collectValues [] tree

Union Types

Union types play a key role in language oriented

programming, as they allow the user to easily

define a tree that will form the abstract syntax

tree of the languagetree of the language

Active Patterns

// definition of the active patternlet (|Bool|Int|Float|String|) input =

// attempt to parse a boollet sucess, res = Boolean.TryParse inputif sucess then Bool(res)else

// attempt to parse an int// attempt to parse an intlet sucess, res = Int32.TryParse inputif sucess then Int(res)else

// attempt to parse a float (Double)let sucess, res = Double.TryParse inputif sucess then Float(res)else String(input)

Active Patterns

// function to print the results by pattern// matching over the active patternlet printInputWithType input =

match input with| Bool b -> printfn "Boolean: %b" b| Int i -> printfn "Integer: %i" i| Int i -> printfn "Integer: %i" i| Float f -> printfn "Floating point: %f" f| String s -> printfn "String: %s" s

// print the results printInputWithType "true"printInputWithType "12"

Active Patterns

Active patterns play a supporting role in

language oriented programming in F#. They

allow the programmer to create abstractions of

complex operations on the abstract syntax treecomplex operations on the abstract syntax tree

Lambda Functions

fun () ->

lovin'()lovin'()

Lambda Functions

Lambda functions are important for language

oriented programming. They allow the users of

the language to embed actions within other

language elementslanguage elements

Custom Operators

let (++) x = x + 1

Custom Operators

Custom operators play a supporting role in

language oriented programming. They allow the

language designer to have a more flexible

syntax.syntax.

Custom Numeric Literals

type Expression =| Constant of int

module NumericLiteralN = let FromZero() = Constant 0let FromZero() = Constant 0let FromOne() = Constant 1let FromInt32 = Constant

let expr = 1N

Custom Numeric Literals

Numeric literals play a supporting role in

language oriented programming. They allow

numeric literals be treated in a more natural

within an embedded languagewithin an embedded language

The Anatomy of a DSLtype Expression =

| Add of Expression * Expression| Subtract of Expression * Expression| Multiply of Expression * Expression| Constant of int| Parameter of stringwith

static member (+) (x, y) = Add(x, y)

Syntax Tree

Combinators

static member (+) (x, y) = Add(x, y)static member (-) (x, y) = Subtract(x, y)static member (*) (x, y) = Multiply(x, y)

module NumericLiteralN = let FromZero() = Constant 0let FromOne() = Constant 1let FromInt32 = Constant

let param = Parameter

The Anatomy of a DSL

let expr = (1N + 2N) * (5N - 2N)

val expr : Expression =Multiply (Add (Constant 1,Constant 2),

Subtract (Constant 5,Constant 2))

The Anatomy of a DSL

• Expressions now have an abstract tree like representation:

― Multiply

― Add

― Constant 1

― Constant 2― Constant 2

― Subtract

― Constant 5

― Constant 2

• This can then be evaluated

• Or we can preform more advanced analysis

The Anatomy of a DSL

let evaluateExpression parameters =let rec innerEval tree =

match tree with| Multiply (x, y) -> innerEval x * innerEval y| Add (x, y) -> innerEval x + innerEval y| Subtract (x, y) -> innerEval x - innerEval y| Constant value -> value| Constant value -> value| Parameter key -> Map.find key parameters

innerEval

let expr = (1N + 2N) * (5N - 2N)

evaluateExpression Map.empty expr

The Anatomy of a DSL

The Anatomy of a DSL

let rec simplifyExpression exp =let simpIfPoss op exp1 exp2 =

let exp' = op (simplifyExpression exp1, simplifyExpression exp2)

if exp' = exp then exp' else simplifyExpression exp'match exp with| Multiply(Constant 0, Constant _) -> Constant 0| Multiply(Constant 0, Constant _) -> Constant 0| Multiply(Constant _, Constant 0) -> Constant 0| Multiply(Constant n1, Constant n2) -> Constant (n1 * n2)| Add(Constant n1, Constant n2) -> Constant (n1 + n2)| Subtract(Constant n1, Constant n2) -> Constant (n1 - n2)| Multiply(exp1, exp2) -> simpIfPoss Multiply exp1 exp2| Add(exp1, exp2) -> simpIfPoss Add exp1 exp2| Subtract(exp1, exp2) -> simpIfPoss Add exp1 exp2| Constant _ | Parameter _ -> exp

The Anatomy of a DSL

Quotations

let quotation = <@ (1 + 2) * (5 - 2) @>

val quotation : Quotations.Expr<int> =

Call (None, Int32 op_Multiply[Int32,Int32,Int32](Int32, Int32),

[Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),[Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),

[Value (1), Value (2)]),

Call (None, Int32 op_Subtraction[Int32,Int32,Int32](Int32, Int32),

[Value (5), Value (2)])])

Quotations

• Allow you to grab a tree structure that

represents a piece of F# code

• This tree structure can then be:• This tree structure can then be:

– Interpreted

– Compiled

– Converted to instructions understood by another

system or runtime

Microsoft Accelerator

• A project from MRS, now available on

Microsoft Connect [1]

• Automatically parallelize code for execution • Automatically parallelize code for execution

on the GPU or x64 multicore

• Implemented as an unmanaged library with

.NET wrapper

[1] http://connect.microsoft.com/acceleratorv2

Microsoft Accelerator

• Based on “Parallel Array”

• You define a set of operations to process the

content of the arraycontent of the array

• The Accelerator runtime will then process the

operations in parallel

Microsoft Accelerator

User Program – Define Operations

AcceleratorAccelerator

Accelerated Program

DX9Target X64Target

Input data Input data

Microsoft Accelerator – Code!

let nums = [| 6; 1; 5; 5; 3 |]let input = new FloatParallelArray(nums); let sum = ParallelArrays.Shift(input, 1) + input +

ParallelArrays.Shift(input, -1); let output = sum / 3.0f;

let target = new DX9Target(); let res = target.ToArray1D(output);

Game of Life

Game of Life

• Green - When an existing cell (green in the middle) has three or two neighbours it survives to the next round

• Red - When an existing cell has less than two neighbours it dies, because it is lonely (first red case), when it has more than three neighbours it dies of overcrowding (second red case)

• Blue - Finally, a new cell is born in an empty grid location if there are exactly three neighbours .

The implementation

1

1 1

1

etc. ...Sum of all

neighbours

initial rotation 1

Game of Life

GPUCPU

Game of Life

/// Evaluate next generation of the life game state

let nextGeneration (grid: Matrix<float32>) =

// Shift in each direction, to count the neighbourslet sum = shiftAndSum grid offsets

// Check to see if we're born or remain alive

GPUCPU

// Check to see if we're born or remain alive(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)

Game of Life

/// Evaluate next generation of the life game state[<ReflectedDefinition>]let nextGeneration (grid: Matrix<float32>) =

// Shift in each direction, to count the neighbourslet sum = shiftAndSum grid offsets

// Check to see if we're born or remain alive

GPUCPU

// Check to see if we're born or remain alive(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)

DEMO

Game of life in F# with Microsoft Accelerator

Thanks!

• A big thanks to Tomáš Petříček!

• More info on his blog:• More info on his blog:– http://tomasp.net/articles/accelerator-intro.aspx

Books

Shameless Plug!

• Robert Pickering’s Beginning F# Workshop:

– Thursday 9th Sept – 10th September 2010

http://skillsmatter.com/course/open-source-http://skillsmatter.com/course/open-source-

dot-net/robert-pickerings-beginning-f-workshop