SPL - PS12

Reactor and Java New-IO (nio) classes

Multi-Threaded server, as seen in Tirgul 11- one thread is in charge of accepting new connections- opens a new thread for each new client. - Thus, handling 30 clients requires 31 threads.

Multi-Threaded server is problematic for several reasons:

It's wastefull - Creating a new Thread is relatively expensive. - Each thread requires a fair amount of memory. - Threads are blocked most of the time waiting for network IO.

It's not scalable - A Multi-Threaded server can't grow to accommodate hundreds of concurrent requests - It's also vulnerable to Denial Of Service attacks

Poor availability - It takes a long time to create a new thread for each new client. Moreover, the response time degrades as the number of clients rises.

The Reactor Pattern is another (better) design for handling several concurrent clients

1. Use Non-Blocking IO, so Threads don't waste time waiting for Network.

2. Have one thread in charge of the Network - accepting new connections and handling IO. As the Network is non-blocking, read, write and accept operations "take no time", and a single thread is enough for all the clients.

3. Have a fixed number of threads (for example in a thread pool), which are in charge of the protocol. These threads perform: message framing (tokenizations) message processing (what to do with each message).

Note - unlike the Multi-Threaded server, in this design a single thread may handle many clients.

Java NIO API – previous PS (last week)

Java's java.nio package is a new, efficient IO package. It also support Non-blocking IO.

The key players we need to know are:

- Channels

- Buffers

- Selector

Java NIO API

Channels, a new primitive I/O abstraction

java.nio.channels.Channel is an interface

Channels (classes implementing the interface) are designed to provide for bulk data transfers to and from NIO buffers (A Channel is something you can read from and write to).

Channels can be either blocking (by default) or non-blocking.

socket channels

allow for data transfer between sockets and NIO buffers.

java.nio.channels.SocketChannel (similar to Socket)java.nio.channels.ServerSocketChannel (similar to ServerSocket)

Here read(), write() and accept() methods can be non-blocking.

The ServerSocketChannel's accpet() method returns a SocketChannel.

The other side of the connection can of course be either blocking or non-blocking (it doesn't matter).

Java NIO API

socket channels

Setting up a non-blocking ServerSocketChannel listening on a specific port:

int port = 9999;ServerSocketChannel ssChannel = ServerSocketChannel.open();ssChannel.configureBlocking(false); // false for non-blockingssChannel.socket().bind( new InetSocketAddress(port) );

Setting up a non-blocking SocketChannel and connecting to a server:

SocketChannel sChannel = SocketChannel.open();sChannel.connect( new InetSocketAddress("localhost", 1234) );sChannel.configureBlocking(false); // false for non-blocking

Java NIO API

NIO buffers

NIO data transfer is based on buffers (java.nio.Buffer and related classes).

A buffer is a container that can hold a finite and contiguous sequence of primitive data types. It's essentially an object wrapper around an array of bytes with imposed limits.

Using the right implementation, allows the buffer contents to occupy the same physical memory used by the operating system for its native I/O operations, thus allowing the most direct transfer mechanism, and eliminating the need for any additional copying.

In most operating systems (provided some properties of the particular area of memory)transfer can take place without using the CPU at all. The NIO buffer is intentionally limited in features in order to support these goals.

In simple words – allows faster and more efficient implementations

NIO buffers maintain several pointers that dictate the function of its accessor methods. The NIO buffer implementation contains a rich set of methods for modifying these pointers: flip() get() put() mark() reset()

Java NIO API

NIO buffers

Channels know how to read and write into Buffers, and buffers can read and write into other buffers.

We'll be using ByteBuffer (These are buffers that hold bytes).

Creating a new ByteBuffer:

final int NUM_OF_BYTES = 1024; ByteBuffer buf = ByteBuffer.allocate(NUM_OF_BYTES);

Creating a ByteBuffer from a an array of bytes:

byte[] bytes = new byte[10]; ByteBuffer buf = ByteBuffer.wrap(bytes);

Java NIO API

Capacity pointer - is the maximum number of items a buffer can holdposition pointer – points to “start"limit pointer – value that ranges from zero to capacity, representing an arbitrary limitation for the buffer

ByteBuffer buffer = ByteBuffer.allocate(512); //creation of a nondirect ByteBufferString str = "Hello";byte[] data = str.getBytes();buffer.put(data); //add the string "Hello" to the byte buffer

The position pointer now points to the next empty cell after the data. So if we are to use this buffer to read the data, we need to flip the position of the position pointer.

buffer.flip(); //readies the buffer for draining by resetting the position and limit pointerint limit = buffer.limit();byte[] data = new byte[limit];buffer.get(data);System.out.println(new String(data));

After the flip() is called, the position pointer points to the first cell, and the limit pointer points to the cell where the position pointer used to point before the flip() method was called.

Java NIO API

NIO buffers

The flip() method

position pointer – points to “start"limit pointer – points to “size" or “end"

The flip() method, moves the position pointer to the origin of the underlying array (if any) and the limit pointer to the former position of the position pointer.

Each buffer has a size and a position marker. A read operation reads a specified number of bytes from the current position, and updates the position marker to point to the yet unread bytes.

Similarly, a write operation writes some bytes from the current position, and then advances the position.

You can't read or write more than the size of the buffer.

This means that if someone wrote some bytes into a buffer, and then you want to read them, you need to set the position marker to the start of the buffer, and the size of the buffer to the former position.

Java NIO API

Character sets

In Java, a character set is a mapping between Unicode characters (or a subset of them) and bytes.

The java.nio.charset package of NIO provides facilities for identifying character sets and providing encoding and decoding algorithms for new mappings.

Charset charset = Charset.forName("UTF-8");_decoder = charset.newDecoder();_encoder = charset.newEncoder();

Java NIO API

From Channel to Buffer and back

Reading from a channel to a buffer:

numBytesRead = _socketChannel.read(buf); //returning number of bytes read

Writing from a buffer to a channel:

numBytesWritten = _socketChannel.write(buf); //returning number of bytes written

If read or write returns -1, it means that the channel is closed.

Read and write operations on Buffers update the position marker accordingly.

Java NIO API - Selectors

When we perform non-blocking IOfor example, read(): - Read the numbers of currently available bytes, and returns. This means we can read or write 0 bytes, and this is what will probably happen most of the time.

- We would like a way for our thread to wait until any of our channels is ready, and only then perform the read(), write() or accept(), only on the ready channel.

A selector (java.nio.channels.Selector and subclasses) provides a mechanism:- for waiting on channels - and recognizing when one or more become available for data transfer.

Java NIO API - Selectors

A selector (java.nio.channels.Selector and subclasses) provides a mechanism:- for waiting on channels - and recognizing when one or more become available for data transfer.

When a number of channels are registered with the selector - it enables blocking of the program flow until at least one channel is ready for use, or until an interruption condition occurs.

Although this multiplexing behavior could be implemented with Java threads, the selector can provide a significantly more efficient implementation using native platform threads or, more likely, even lower-level operating system constructs. A POSIX-compliant operating system, for example, would have direct representations of these concepts, select(). A notable application of this design would be the common paradigm in server software which involves simultaneously waiting for responses on a number of sessions.

Java NIO API - Selectors

The Selector registers itself to a Channel- Registration means that the channel informs the Selector when some event happens.

This event can be:- the channel is ready to be read- the channel can now be written to- the channel can now accept()

- During registration, the channel is told which events to monitor.

- In addition, when registering a Selector to a channel, one can add an "attachment" - An attachment is an object that we will have access to when the event occurs (usually, this object is associated with a task that should be performed).

Registration:Selector selector = Selector.open(); // a new SelectorObject anAttachment = new Object();socketChannel.register(selector, SelectionKey.OP_READ, anAttachmemt); //SelectionKey.OP_READ is a number representing the READ event

Java NIO API

select() method

- This method blocks until at least one of the channels is ready for the registered event.

- Then, a list of SelectionKeys is returned.

- Each Selectionkey is associated with one event, and holds the attachment registered with the event.

Java NIO API - The Reactor

"core" reactor classes: - Reactor- ConnectionHandler- ConnectionAcceptor- ProtocolTask

- The Reactor is the main thread, which is in charge of reading, writing and accepting new connections. - Reading and Writing are encapsulated in the ConnectionHandler class- accepting is encapsulated in the ConnectionAcceptor class

- The reactor thread also holds a thread-pool of workers (an Executor). These worker-threads are in charge of the protocol handling. they receive the bytes read from the network, and let the protocol do something with them. The class which represents tasks submitted to the executor is ProtocolTask.

- The ProtocolTask object is created along with the ConnectionHandler, and the same task object is reused over and over again. This is a bit different than what you saw in the beginning of the course, where each task was run once and then discarded.

Java NIO API - The Reactor

Once some bytes were read by the reactor thread - The ConnectionHandler adds these bytes to its ProtocolTask, and passes the task to the thread pool.

- The ProtocolTask passes the bytes to the tokenizer, which looks for complete messages.

- Then, the protocolTask goes over all the messages which are available in the tokenizer, and asks the protocol to process them.

- After the protocol process a message and return a response, the tokenizer is used to add framing information to the response, and then convert it into bytes.

- Then, the protocol-task thread adds the response bytes back to the ConnectionHandler.

- Notice - we don't have one thread per connection. - We have one ConnectionHandler object per connection.

This object holds: the socket, the task and the protocol and tokenizer instances associated with this connection.

Java NIO API - The Reactor

- Also notice - we are using an extended version of the SevrerProtocol (presented in the previous practical session). This extended version (AsyncServerProtocol) has two additional functions:

1. boolean shouldClose() - - returns true after the terminating message has been encountered. - This is used to signal the connection handler that the protocol has done its work and it's time to send all the remaining bytes and quit.

2. void connectionTerminated() - used to notify the protocol that the client closed the connection, or that the network connection was lost. This is called by the ConnectionHandler when it identifies a network error.

The thread of the reactor performs the following:

1. Create a new thread pool (an executor). 2. Create a new Selector. 3. Create a new ServerSocketChannel, and bind it to a port. 4. Create a new ConnectionAcceptor 5. Register the ServerSocketChannel in the Selector, asking for accept readiness. While(_shouldRun && selector.isOpen()) 6. wait for notifications from the selector. //selector.select() Get list of selection keys with pending events //selector.selectedKeys().iterator();

For each notification (SelectionKey) arrived check: //while (it.hasNext())

Accept notification - the server socket is ready to accept a new connection so: Get attached ConnectionAcceptor 1. call accept (2. Now a new socket was created): 3. Get a new channel for the connection request 4. register new socket in the Selector (5. return SelectionKey) 6. Create new ConnectionHandler and initialize it: //unique per connection 7. create protocol 8. create tokenizer 9. Create ProtocolTask //unique per connection 10. Attach itself (this.) to selectionKey 11. set the handler to read only mode

Read notification - For each socket which is ready for reading: Get attached ConnectionHandler 1. read() 2. read some bytes (to buffer from the socket channel) and pass them down to the protocolTask: 3. addBytes(buf) 4. execute(pt) (run pt using the thread pool): //not in the main thread 5. add all the bytes we have to the tokenizer go over all complete messages (if ready) and process them: 6. msg = nextMessage() using tokenizer 7. processMessage(msg) using protocol, get response If there is a response: 8. send using ConnectionHandler 9. set the handler to Read or Write Mode Write notification - For each socket which is ready for writing: Get attached ConnectionHandler 1. write() //runs in the main thread if all the data has been successfully sent (outData.size() == 0): 2. set the handler to read only mode (and return) if there is something to send 3. write some bytes (to socket channel from buffer)

The thread of the reactor performs the following:

1. Create a new thread pool (an executor). 2. Create a new Selector. 3. Create a new ServerSocketChannel, and bind it to a port. 4. Create a new ConnectionAcceptor 5. Register the ServerSocketChannel in the Selector, asking for accept readiness.

While(_shouldRun && selector.isOpen()) 6. wait for notifications from the selector. //selector.select() Get list of selection keys with pending events //selector.selectedKeys().iterator();

For each notification (SelectionKey) arrived check:

The thread of the reactor performs the following:

1. Create a new thread pool (an executor). ExecutorService executor = Executors.newFixedThreadPool(_poolSize);

2. Create a new Selector. selector = Selector.open();

3. Create a new ServerSocketChannel, and bind it to a port. ssChannel = createServerSocket(_port); ServerSocketChannel ssChannel = ServerSocketChannel.open();

ssChannel.configureBlocking(false); ssChannel.socket().bind(new InetSocketAddress(port));

4. Create a new ConnectionAcceptor ConnectionAcceptor connectionAcceptor = new ConnectionAcceptor( ssChannel, _data);

5. Register the ServerSocketChannel in the Selector, asking for accept readiness. ssChannel.register(selector, SelectionKey.OP_ACCEPT, connectionAcceptor);

While(_shouldRun && selector.isOpen()) 6. wait for notifications from the selector - selector.select(); Get list of selection keys with pending events: Iterator it = selector.selectedKeys().iterator();

For each notification (SelectionKey) arrived check: while (it.hasNext()) { SelectionKey selKey = (SelectionKey) it.next(); // Remove the selection key from the list to indicate that it is being // processed. it.remove() removes the last item returned by next. it.remove();

Accept notification - the server socket is ready to accept a new connection so: Get attached ConnectionAcceptor 1. call accept (2. Now a new socket was created): 3. Get a new channel for the connection request 4. register new socket in the Selector (5. return SelectionKey) 6. Create new ConnectionHandler and initialize it: 7. create protocol 8. create tokenizer 9. Create ProtocolTask 10. Attach itself (this.) to selectionKey 11. set the handler to read only mode

Accept notification - the server socket is ready to accept a new connection so: if (selKey.isValid() && selKey.isAcceptable()) { Get attached ConnectionAcceptor ConnectionAcceptor acceptor = (ConnectionAcceptor) selKey.attachment(); 1. call accept (2. Now a new socket was created): acceptor.accept();

3. Get a new channel for the connection request SocketChannel sChannel = _ssChannel.accept();

4. register new socket in the Selector (5. return SelectionKey) SelectionKey key = sChannel.register(_data.getSelector(), 0);

6. Create new ConnectionHandler and initialize it: ConnectionHandler handler = ConnectionHandler.create(sChannel, _data, key);

7. create protocol _protocol = _data.getProtocolMaker().create();

8. create tokenizer _tokenizer = _data.getTokenizerMaker().create();

9. Create ProtocolTask _task = new ProtocolTask(_protocol, _tokenizer, this);

10. Attach itself (this.) to selectionKey _skey.attach(this); //SelectionKey _skey;

11. set the handler to read only mode handler.switchToReadOnlyMode();

Read notification - For each socket which is ready for reading: Get attached ConnectionHandler 1. read() 2. read some bytes (to buffer from the socket channel) and pass them down to the ProtocolTask: 3. addBytes(buf) 4. execute(pt) (run pt using the thread pool- DIFFERENT THREAD): 5. add all the bytes we have to the tokenizer go over all complete messages (if ready) and process them: 6. msg = nextMessage() using tokenizer 7. processMessage(msg) using protocol, get response If there is a response: 8. send using ConnectionHandler 9. set the handler to Read or Write Mode

Read notification - For each socket which is ready for reading: if (selKey.isValid() && selKey.isReadable()) {

Get attached ConnectionHandler ConnectionHandler handler = (ConnectionHandler) selKey.attachment();

1. read() handler.read();

2. read some bytes (to buffer from the socket channel) and numBytesRead = _sChannel.read(buf);

pass them down to the ProtocolTask: 3. addBytes(buf) buf.flip(); _task.addBytes(buf);

4. execute(pt) (run pt using the thread pool- DIFFERENT THREAD): _data.getExecutor().execute(_task);

5. add all the bytes we have to the tokenizer this._tokenizer.addBytes(buf);

go over all complete messages (if ready) and process them: while (_tokenizer.hasMessage()) {

6. String msg = _tokenizer.nextMessage(); 7. String response = this._protocol.processMessage(msg); If there is a response: 8. send using ConnectionHandler this._handler.addOutData(bytes); //_outData.add(buf);

9. set the handler to Read or Write Mode switchToReadWriteMode();

public class ProtocolTask implements Runnable {private final ServerProtocol _protocol;private final StringMessageTokenizer _tokenizer;private final ConnectionHandler _handler;

/** * The fifo queue, which holds data coming from the socket. Access to the * queue is serialized, to ensure correct processing order - even if more

* data is received by the reactor while previous data is still being processed. */private final Vector<ByteBuffer> _buffers = new Vector<ByteBuffer>();

public ProtocolTask(final ServerProtocol protocol, final StringMessageTokenizer tokenizer, final ConnectionHandler h) {this._protocol = protocol;this._tokenizer = tokenizer;this._handler = h;

}

// We synchronize on ourselves, in case we are executed by several threads from the thread pool.public synchronized void run() {

synchronized (_buffers) { // first, add all the bytes we have to the tokenizer while(_buffers.size() > 0) { ByteBuffer buf = _buffers.remove(0); this._tokenizer.addBytes(buf); } }

// now, go over all complete messages and process them. while (_tokenizer.hasMessage()) { String msg = _tokenizer.nextMessage(); String response = this._protocol.processMessage(msg); if (response != null) { try { ByteBuffer bytes = _tokenizer.getBytesForMessage(response); this._handler.addOutData(bytes); } catch (CharacterCodingException e) { e.printStackTrace(); } } }

}

// This is invoked by the ConnectionHandler which runs in the reactor main thread.public void addBytes(ByteBuffer b) {

// We synchronize on _buffers and not on "this" because // run() is synchronized on "this", and it might take a long time to run. synchronized (_buffers) {

_buffers.add(b); }

}}

Write notification - For each socket which is ready for writing: Get attached ConnectionHandler 1. write() if all the data has been succesfully sent (outData.size() == 0): 2. set the handler to read only mode (and return) if there is something to send 3. write some bytes (to socket channel from buffer)

Write notification - For each socket which is ready for writing: if (selKey.isValid() && selKey.isWritable()) {

Get attached ConnectionHandler ConnectionHandler handler = (ConnectionHandler) selKey.attachment();

1. write() handler.write();

if all the data has been succesfully sent (outData.size() == 0): 2. set the handler to read only mode (and return)

if there is something to send 3. write some bytes (to socket channel from buffer)

protected Vector<ByteBuffer> _outData = new Vector<ByteBuffer>(); // If there is something to send - send the first byte buffer // We will return to this write() operation very soon because the selector // will keep firing the OP_WRITE event after we are done writing this buffer // and check if there are more buffers to be sent. ByteBuffer buf = _outData.remove(0); //Removes element at position 0 in Vector. if (buf.remaining() != 0) { try { _sChannel.write(buf); } catch (IOException e) { // This should never happen because the selector told us we are in Writable mode. e.printStackTrace() } // Check if the buffer contains more data: we could not send all // of the buffer in one write (the output buffer of the socket got full). // So we remember that there is more data to be sent. // We will receive a new OP_WRITE event when the output buffer of the socket // is not full anymore and complete the write operation then. if (buf.remaining() != 0) { _outData.add(0, buf); //insert element at position 0 in Vector } }