Season of mists and mellow fruitfulness,

Close bosom-friend of the maturing sun;

Conspiring with him how to load and bless

With fruit the vines that round the thatch-eves run;

To bend with apples the moss'd cottage-trees,

And fill all fruit with ripeness to the core;

To swell the gourd, and plump the hazel shells

With a sweet kernel; to set budding more,

And still more, later flowers for the bees,

Until they think warm days will never cease,

For Summer has o'er-brimm'd their clammy cells.

Who hath not seen thee oft amid thy store?

Sometimes whoever seeks abroad may find

Thee sitting careless on a granary floor,

Thy hair soft-lifted by the winnowing wind

Or on a half-reap’d furrow sound asleep,

Drowsed with the fume of poppies, while thy hook

Spares the next swath and all its twinèd flowers:

And sometimes like a gleaner thou dost keep

Steady thy laden head across a brook;

Or by a cyder-press, with patient look,

Thou watchest the last oozings, hours by hours.

Where are the songs of Spring? Ay, where are they?

Think not of them, thou hast thy music too,—

While barrèd clouds bloom the soft-dying day

And touch the stubble-plains with rosy hue;

Then in a wailful choir the small gnats mourn

Among the river-sallows, borne aloft

Or sinking as the light wind lives or dies;

sinking

And full-grown lambs loud bleat from hilly bourn;

Hedge-crickets sing; and now with treble soft

The redbreast whistles from a garden-croft;

And gathering swallows twitter in the skies.

Everything You Wanted to Know

About Writing Async, Concurrent HTTP Apps

in Java

Agenda

Mostly this:

Agenda

And this:

Agenda

And this:

About your speaker@jbaruchgithub.com/jbaruch

linkd.in/jbaruch

stackoverflow.com/users/402053/jbaruch

What Frog?

What Frog?

What Frog?

What Frog?

Requirements– parallel file Downloads– Parallel file parts– interrupt/pause/resume– Progress events– Checksums caching

First Association for “concurrent downloader”

Lucky day: Download manager written in java!

Let’s look if we can use it!

1.No traceable license2.No website or docs3.No traceable sources4. It’s an app, not a lib

Java.net.urlconnection1. Memory wasteful (buffering)2. Minimal API3. Blocking streams

What we’re looking for1. Async/non-blocking2. Event callbacks

What is IT going to take1. Reactor2. nio

Welcome to the reactor

– pattern for lightweight concurrency

– Event driven– Threads reuse– Uses non-blocking Io

Original pattern

http://www.dre.vanderbilt.edu/~schmidt/PDF/reactor-siemens.pdf

Guess the author by the diagram

http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf

In Java, Reactor

means NIO

Selector as a multiplexer

Java version - RegisteringSocketChannel channel= SocketChannel.open();

socketChannel.connect(new

InetSocketAddress("http://remote.com", 80));

...

Selector selector = Selector.open();

channel.configureBlocking(false);

SelectionKey k = channel.register(selector, SelectionKey.OP_READ);k.attach(handler);

Java version - Dispatcherwhile (!Thread.interrupted()) {

selector.select();Set selected = selector.selectedKeys();Iterator it = selected.iterator();while (it.hasNext()) SelectionKey k = (SelectionKey)(it.next(); ((Runnable)(k.attachment())).run(); selected.clear();

}

Handling reactor events is complex

–Need to maintain state–Buffering – assembling chunks

–Coordinating async events

Nio libraries–Most of them are servers

–Netty, grizzly, etc.–Apache Mina–Apache HTTP components asyncclient

–Ning http client

–Client and server nio library–Evolved from netty–Latest release October 2012

Nio libraries

Ning’s async http client

Here it is!

try (AsyncHttpClient asyncHttpClient = new AsyncHttpClient()) { ListenableFuture<Response> future = asyncHttpClient.prepareGet( "http://oss.jfrog.org/api/system/ping").execute( new AsyncCompletionHandler<Response>() { @Override public Response onCompleted(Response response) { System.out.println(response.getResponseBody()); return response; }

@Override public void onThrowable(Throwable t) { t.printStackTrace(); } }); Response response = future.get();}

HAC Concepts–Request producer–Response consumer

Choosing between ning and http asyncclient

"All problems in computer science can be solved by another level of indirection"

David Wheeler

public interface HttpProviderDownloadHandler {

void onResponseReceived(int statusCode, Map<String, List<String>> headers);

boolean onBytesReceived(ByteBuffer buf);

void onFailed(Throwable error);

void onCanceled();

void onCompleted();}

Head to head

Feature/Library Ning client Http Async ClientMaturity Good Very new (early 2014)Download cancelation

Easy With bugs

Progress hooks

Events not granular enough

Just use onByteReceived()

Documentation A bit sparse MinimalServer-side counterpart

None, client only

org.apache.httpcomponentshttpcore-nio

Performance?

http://blogs.atlassian.com/2013/07/http-client-performance-io/

Rfc2616: a universe of its own

Confused?

Just read some stackoverflow(and improve your rep as you go)

And that one for

discovering that range

header is lost on redirect

Question!

What should be content-length when using compression?

https://github.com/http2/http2-spec/issues/46

Question!

Why when redirected to CDN all the chunks start from zero?

HttpAsyncClientBuilder builder = HttpAsyncClients.custom();// add redirect strategy that copies "range" headers, if existbuilder.setRedirectStrategy(new DefaultRedirectStrategy() {

@Overridepublic HttpUriRequest getRedirect(HttpRequest request, HttpResponse

response, HttpContext context)HttpUriRequest redirectRequest = super.getRedirect(request,

response, context);// copy "Range" headers, if existHeader[] rangeHeaders = request.getHeaders(HttpHeaders.RANGE);

if (rangeHeaders != null) {for (Header header : rangeHeaders) {

redirectRequest.addHeader(header);}

}return redirectRequest;

}});

Question!

How many simultaneous connections should I open?

Question!

What’s wrong with the following code?

public static String encodeUrl(String urlStr) { URLEncoder.encode(urlStr, "UTF-8"); ...}

Decoded URLs cannot bere-encoded to the same form

http://example.com/?query=a&b==c

Cannot be decoded back after it was encoded:

http://example.com/?query=a%26b==c

Don’t use java.net.URLEncoder

“Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x-www-form-urlencoded MIME format.For more information about HTML form encoding, consult the HTML specification.”

AHC Alternatives

Question!

How do I close a socket correctly?

How hard can it be to close a socket?

The art of socket closing

http://www.safaribooksonline.com/library/view/http-the-definitive/1565925092/ch04s07.html

Half-closed: no new customers

Never block in socket close()

The other side expects you to clean up nicely

It will give up on time outYou will wait (forever)

Remember?

Question!

How can I write file parts concurrently?

– Write to separate files, combine on finish

– Write to same file, seeking to the right position

private FileLock lock(FileChannel fileChannel) throws IOException { FileLock lock = fileChannel.tryLock(Long.MAX_VALUE - 1, 1, false); if (lock == null) { throw new OverlappingFileLockException(); } return lock;}

OS Level File Locking –Advisory exclusive

WTF?!

VM Level File Locking

VM Level File Locking–Prevent same VM threads writing to the file when we started closing it

–Closing sequence:–Release file locks–Close channels–Rename a file to it's final name (remove .part)

–Erase progress info

VM Level File LockingReentrantReadWriteLock.ReadLock writeToFileLock = rwl.readLock();ReentrantReadWriteLock.WriteLock closeFileLock = rwl.writeLock();

public void close() throws IOException { this.closeFileLock.lock();}

public int write(int partIndex, ByteBuffer buf) { if (!this.writeToFileLock.tryLock()) { throw new IllegalStateException("File is being closed"); } ...}

What’s next?

http/2– Mostly standardizing Google's

spdy– Header compression– multiplexing– Prioritization– Server push – On the way clear some stuff– E.g. compressed content length

Ease the load

Links!

RTFM: RFC 2616Ultimate book: HTTP: The Definitive Guide

AmazonSafari

Reactor patternDoug Lea on NIO

No, Thank you!

A Quick Tourof Logos

The Logical Appeal

So what exactly is logic? Who cares?Informally, logic is about saying things that make sense. You can think of it in that way if you like.

“It's pretty sunny today, so you should wear sunscreen.”

Formally, logic is the art of arguing – not like a fight or debate, but by using the information we already

know to draw new and useful conclusions.

“If it's sunny today, you should wear sunscreen.Indeed it is sunny today.

Therefore, you should wear sunscreen.”

“But wait. That just looked like the exact same thing you said before, you hack.”

Well, yes. But that's how an argument looks in standardform! You can break down any argument into this form;that makes it easier to think about.

If it's sunny today, you should wear sunscreen.

It is sunny today.___________________________________________

Therefore, you should wear sunscreen.

Premises/givens

Conclusion(Premises always come first, and the conclusion always comes last.)

How about a more complex argument?

1. This piece of fresh fruit is fuzzy.2. It also has seeds.

3. If a fruit is fuzzy, it's either a kiwi or a peach.4. Peaches have a pit; they don't have seeds.

5. So the fruit can't be a peach.6. So the fruit must be a kiwi.

Which of the above sentences is a conclusion?

How about a more complex argument?

1. This piece of fresh fruit is fuzzy.2. It also has seeds.

3. If a fruit is fuzzy, it's either a kiwi or a peach.4. Peaches have a pit; they don't have seeds.

5. So the fruit can't be a peach.6. So the fruit must be a kiwi.

Good logic lets us cobble together lots ofdifferent pieces of information, and tell from

them what's probably or definitely true.

But what counts as “good logic”?

That argument was good (made sense), because the conclusion followed from the premises. We'll see what this means in a moment.

Why don't we look at a bad argument?

But what counts as “good logic”?

Some people have fallen off cliffs and lived.

Therefore, if I jump off this cliff, I will definitely be fine.

“Come on.What couldpossibly gowrong?”

But what counts as “good logic”?

Some people have fallen off cliffs and lived.

Therefore, if I jump off this cliff, I will definitely be fine.

This argument is weak. Although the premise is true,it's easy to think of ways (very painful ways) that theconclusion could be false. The easiest way to spot badlogic is to do just that: try to think of another way out.

(Philosophers call these counterexamples).

But what counts as “good logic”?

Let's look at two kinds of arguments.1. Deductive reasoning: All interns can breathe fire. So Philip

can breathe fire.

Is there a piece of the puzzle missing?

But what counts as “good logic”?

Let's look at two kinds of arguments.1. Deductive reasoning: All interns can breathe fire. Philip is an

intern. So Philip can breathe fire.

Sometimes you may encounter “hidden” statementsand ideas, which the writer sneaks in but doesn'tsay outright.

But what counts as “good logic”?

Let's look at two kinds of arguments.1. Deductive reasoning:

All interns can breathe fire. Philip is an intern. So Philip can breathe fire.

Are the premises true?If so, then the conclusion's100% guaranteed true.No getting around it!

But what counts as “good logic”?

Let's look at two kinds of arguments.2. Inductive reasoning:

I touched a stove and it burned me. I did this fifty times, and the same thing happened. So

the next time I touch the stove, it will burn me.

Are the premises true?If so, then the conclusion'sprobably true. There mightstill be exceptions.

LIKE WHAT?

How is this useful to me?

Like Mr. Morgan said, the ability to make strong logical arguments will become more and more important later on in high school and college.

Pathos and ethos are still valuable! But your audience will be a lot better at questioning them. Logos is handy because, if you use it well, it can't really be disproved.

How is this useful to me?But even better is the superpower to spot weak logic.

Next time you watch TV or go online (with your parents' permission, of course), try to keep track of how many different arguments are being pitched to you by ads. How much info is given to you? How much is left out?

What time is it?Adven- wait, no. Activity time!

Pair off into groups of four. Each group will receive an exampleof a poor argument (these may be either inductive or deductive).

With your group, you will have 5 minutes to try to come up withone counterexample - one way in which the argumentcould be wrong, even if the premises are definitely true.Poke it full of holes!Also, choose a group representative to tell us your reasoning.

(It's OK to imagine unlikely or weird explanations;don't be afraid to think outside the box.Oddly enough, logic has very little to do with facts.)

Some examples:ARGUMENT: “I pulled an all-nighter studying for last week's bigtest, and I ended up with an A. Tiredness must make me smart!”

COUNTER: What if you got an A because you actually studied?Or maybe the test was going to be easy for you all along?

ARGUMENT: “If I play with Dad's power tools, he'll yell at me.But Dad is yelling at me for something. So I guess I must haveplayed with the power tools.”

COUNTER: What if he's yelling at you for a different reason:scratching the car, or hammer-throwing the cat onto the roof?

Season of mists and mellow fruitfulness,

Close bosom-friend of the maturing sun;

Conspiring with him how to load and bless

With fruit the vines that round the thatch-eves run;

To bend with apples the moss'd cottage-trees,

And fill all fruit with ripeness to the core;

To swell the gourd, and plump the hazel shells

With a sweet kernel; to set budding more,

And still more, later flowers for the bees,

Until they think warm days will never cease,

For Summer has o'er-brimm'd their clammy cells.

Who hath not seen thee oft amid thy store?

Sometimes whoever seeks abroad may find

Thee sitting careless on a granary floor,

Thy hair soft-lifted by the winnowing wind

Or on a half-reap’d furrow sound asleep,

Drowsed with the fume of poppies, while thy hook

Spares the next swath and all its twinèd flowers:

And sometimes like a gleaner thou dost keep

Steady thy laden head across a brook;

Or by a cyder-press, with patient look,

Thou watchest the last oozings, hours by hours.

Where are the songs of Spring? Ay, where are they?

Think not of them, thou hast thy music too,—

While barrèd clouds bloom the soft-dying day

And touch the stubble-plains with rosy hue;

Then in a wailful choir the small gnats mourn

Among the river-sallows, borne aloft

Or sinking as the light wind lives or dies;

sinking

And full-grown lambs loud bleat from hilly bourn;

Hedge-crickets sing; and now with treble soft

The redbreast whistles from a garden-croft;

And gathering swallows twitter in the skies.