ASYNC FRONTIERS

HI, I’M DOMENIC

STANDARDS AS PROTOCOLS

A PROMISE-RETURNING PLATFORM▪ fetch(…)

▪ navigator.mediaDevices.getUserMedia(…)

▪ fontFace.load()

▪ httpResponseBody.text()

▪ navigator.requestMIDIAccess()

▪ serviceWorkerClient.openWindow(…)

▪ crypto.subtle.encrypt(…)

▪ animation.finished

▪ …

THERE’S MORE TO LIFE THAN PROMISES

singular pluralsync values iteratorsasync promises ???

singular pluralsync values iteratorsasync promises ???

THE REAL WORLD IS MORE COMPLICATED

TWO AXES OF INTERESTINGNESS

▪Asynchronicity▪Pluralness

THE ASYNCHRONOUS FRONTIER

SINGULAR/ASYNCQuadrant 2

TAMING COMPLEXITY VIA OPINIONS▪ Promises are eager, not lazy

▪ Continuables a.k.a. “thunks” are a lazy alternative

▪ Instead, we use functions returning promises for laziness

▪ Promises enforce async-flattening▪ No promises for promises

▪ Troublesome for type systems, but convenient for developers

▪ Promise bake in exception semantics▪ Could delegate that to another type, e.g. Either<value, error>

▪ But promise semantics integrate well with JS (see async/await)

RESULT, NOT ACTION

Promises represent the eventual result of an async action; they are not a representation of the action itself

PROMISES ARE MULTI-CONSUMER// In one part of your codeawesomeFonts.ready.then(() => { revealAwesomeContent(); scrollToAwesomeContent();});

// In another part of your codeawesomeFonts.ready.then(() => { preloadMoreFonts(); loadAds();});

CANCELABLE PROMISES (“TASKS”)▪ Showing up first in the Fetch API

▪ A promise subclass

▪ Consumers can request cancelation

▪ Multi-consumer branching still allowed; ref-counting tracks cancelation requests

CANCELABLE PROMISES IN ACTIONconst fp = fetch(url);fp.cancel(); // cancel the fetch

const fp2 = fetch(url);const jp = fp2.then(res => res.json());jp.cancel(); // cancel either the fetch or the JSON body parsing

const fp3 = fetch(url);const jp = fp3.then(res => res.json());const hp = fp3.then(res => res.headers);

jp.cancel(); // nothing happens yethp.cancel(); // *now* the fetch could be canceled

CANCELABLE PROMISES IN ACTION

// Every promise gets finallyanyPromise.finally(() => { doSomeCleanup(); console.log("all done, one way or another!");});

// Canceled promises *only* get finally triggeredcanceled .then(() => console.log("never happens")) .catch(e => console.log("never happens")) .finally(() => console.log("always happens"));

CANCELABLE PROMISES IN ACTION

// All together now:

startSpinner();

const p = fetch(url) .then(r => r.json()) .then(data => fetch(data.otherUrl)) .then(r => r.text()) .then(text => updateUI(text)) .catch(err => showUIError()) .finally(stopSpinner);

cancelButton.onclick = () => p.cancel();

DESIGN DECISIONS FOR TASKS▪ Ref-counting, instead of disallowing branching

▪ Cancelation (= third state), instead of abortion (= rejected promise)

▪ Subclass, instead of baking it in to the base Promise class

▪ Evolve promises, instead of creating something incompatible

TASKS ARE IN PROGRESS,BUT I AM HOPEFUL

PLURAL/SYNCQuadrant 3

READ-ONLY VS. MUTABLE

▪ Iterator interface▪ next() → { value, done }

▪ Generator interface▪ next(v) → { value, done }

▪ throw(e)

▪ return(v)

ITERATORS ARE SINGLE-CONSUMER▪ Once you next() an iterator, that value is gone forever

▪ If you want a reusable source of iterators, have a function that returns fresh iterators▪ Iterables have [Symbol.iterator]() methods that do exactly that

▪ (… except sometimes they don’t)

▪ Is the iterator or the iterable fundamental?▪ Is the sequence repeatable?

WHERE DO WE ADD METHODS?▪ myArray.map(…).filter(…).reduce(…) is nice

▪ Can we have that for iterators? Iterables?▪ Iterators conventionally share a common prototype (but not always)

▪ Iterables do not—could we retrofit one? Probably not…

▪ If we add them to iterators, is it OK that mapping “consumes” the sequence?

▪ If we add them to iterables, how do we choose between keys/entries/values?

REVERSE ITERATORS?▪ You can’t in general iterate backward through an iterator

▪ The iterator may be infinite, or generated from a generator function, …

▪ You can reduce, but not reduceRight

▪ https://github.com/leebyron/ecmascript-reverse-iterable

ITERATORS WORK,BUT HAVE ROOM TO GROW

PLURAL/ASYNCQuadrant 4!

WHAT NEEDS ASYNC PLURAL, ANYWAY?▪ I/O streams

▪ Directory listings

▪ Sensor readings

▪ Events in general

▪ …

AN ITERATOR OF PROMISES? NO

iop.next() // { value: a promise, done: ??? }iop.next() // { value: a promise, done: ??? }iop.next() // { value: a promise, done: ??? }iop.next() // { value: a promise, done: ??? }

ASYNC ITERATORS

https://github.com/zenparsing/async-iteration/

ai.next() // promise for { value, done }ai.next() // promise for { value, done }ai.next() // promise for { value, done }

ASYNC ITERATORS

https://github.com/zenparsing/async-iteration/

async for (const x of ai) { console.log(x);}

ASYNC ITERATORS

https://github.com/zenparsing/async-iteration/

async function* directoryEntries(path) { const dir = await opendir(path);

try { let entry; while ((entry = await readdir(dir)) !== null) { yield entry; } } finally { yield closedir(dir); }}

THE ASYNC ITERATOR DESIGN▪ The natural composition of sync iterators and promises; fits well with JS

▪ async for-of

▪ async generators

▪ Single-consumer async iterator, plus async iterables = async iterator factories (just like for sync)

▪ Consumer-controlled rate of data flow▪ Request queue of next() calls

▪ Automatic backpressure

▪ Implicit buffering allows late subscription

A SPECIAL CASE OF ASYNC ITERATORS:I/O STREAMS

READABLE STREAMS

const reader = readableStream.getReader();

reader.read().then( ({ value, done }) => { if (done) { console.log("The stream is closed!"); } else { console.log(value); } }, e => console.error("The stream is errored!", e));

https://streams.spec.whatwg.org/

READABLE STREAMS

async for (const chunk of readableStream) { console.log(chunk);}

https://streams.spec.whatwg.org/

READABLE STREAMS

https://streams.spec.whatwg.org/

readableStream .pipeThrough(transformStream1) .pipeThrough(transformStream2) .pipeTo(writableStream) .then(() => console.log("All data successfully written!")) .catch(e => console.error("Something went wrong!", e));

THE READABLE STREAM DESIGN▪ Focused on wrapping I/O sources into a stream interface

▪ Specifically designed to be easy to construct around raw APIs like kernel read(2)

▪ A variant, ReadableByteStream, with support for zero-copy reads from kernel memory into an ArrayBuffer.

▪ Exclusive reader construct, for off-main-thread piping

▪ Integrate with writable streams and transform streams to represent the rest of the I/O ecosystem

▪ Readable streams are to async iterators as arrays are to sync iterators

READABLE STREAMS IN ACTION

ASYNC ITERATORS ARE “PULL”

WHAT ABOUT “PUSH”?

REMEMBER OUR SCENARIOS▪ I/O streams

▪ Directory listings

▪ Sensor readings

▪ Events in general

▪ …

OBSERVABLES

const subscription = listenFor("click").subscribe({ next(value) { console.log("Clicked button", value) }, throw(error) { console.log("Error listening for clicks", e); }, return() { console.log("No more clicks"); }});

cancelButton.onclick = () => subscription.unsubscribe();

https://github.com/zenparsing/es-observable

THE OBSERVABLE DESIGN▪ An observable is basically a function taking { next, return, throw }

▪ The contract is that the function calls next zero or more times, then one of either return or throw.

▪ You wrap this function in an object so that it can have methods like map/filter/etc.

▪ Observables are to async iterators as continuables are to promises

▪ You can unsubscribe, which will trigger a specific action

▪ Push instead of pull; thus not suited for I/O▪ No backpressure

▪ No buffering

OBSERVABLE DESIGN CONFUSIONS▪ Has a return value, but unclear how to propagate that when doing

map/filter/etc.

▪ Tacks on cancelation functionality, but unclear how it integrates with return/throw, or really anything

▪ Sometimes is async, but sometimes is sync (for “performance”)

PULL TO PUSH IS EASY

try { async for (const v of ai) { observer.next(v); }} catch (e) { observer.throw(e);} finally { observer.return();}

PUSH TO PULL IS HARDToo much code for a slide, but:

▪ Buffer each value, return, or throw from the pusher

▪ Record each next() call from the puller

▪ Correlated them with each other:▪ If there is a next() call outstanding and a new value/return/throw comes in, use it

to fulfill/reject that next() promise

▪ If there are extra value/return/throws in the buffer and a new next() call comes in, immediately fulfill/reject and shift off the buffer

▪ If both are empty, wait for something to disturb equilibrium in either direction

IT’S UNCLEAR WHETHER OBSERVABLES PULL THEIR OWN WEIGHT

BEHAVIORSconst gamepads = navigator.getGamepads();const sensor = new sensors.AmbientLight({ frequency: 100 });const bluetooth = doABunchOfWebBluetoothStuff();

requestAnimationFrame(function frame() { console.log(gamepads[0].axes[0], gamepads[0].axes[1]); console.log(gamepads[0].buttons[0].pressed);

console.log(sensor.value);

console.log(bluetooth.value);

requestAnimationFrame(frame);});

BEHAVIOR DESIGN SPACE▪ Continuous, not discrete (in theory)

▪ Polling, not pulling or pushing

▪ The sequence is not the interesting part, unlike other async plurals

▪ Operators like map/filter/etc. are thus not applicable▪ Maybe plus, minus, other arithmetic?

▪ min(…behaviors), max(…behaviors), …?

▪ What does a uniform API for these even look like?

▪ Could such an API apply to discrete value changes as well?

THINK ABOUT BEHAVIORS MORE

WRAPPING UP

singular plural

sync

async

values

Eitherexceptions

primitives iterators

iterables

generatorsreverse iterators

arrays

continuables

promisestasks

async iteratorsstreams

observablesbehaviors

event emitters

THINGS TO THINK ABOUT▪ Representing the action vs. representing the result

▪ Lazy vs. eager

▪ Push vs. pull vs. poll

▪ Mutable by consumers vs. immutable like a value

▪ Allow late subscription or not

▪ Multi-consumer vs. single-consumer

▪ Automatic backpressure vs. manual pause/resume

▪ Micro-performance vs. macro-performance

CHECK OUT GTOR

https://github.com/kriskowal/gtor/

SHOULD YOU EVEN CARE?

NONE OF THIS IS URGENT,EXCEPT WHEN IT IS

THIS IS THEASYNC FRONTIER, AND YOU ARE THE

PIONEERS

Thank you!

@domenic