Is the following code snippet non-blocking? [on hold]

In Javadoc of ByteByffer.compact method, the following code is given:

buf.clear(); // Prepare buffer for use
while (in.read(buf) >= 0 || buf.position != 0) {
buf.flip();
out.write(buf);
buf.compact(); // In case of partial write
}


The thing is that, while it works, it suffers from the following problem, at least in my understanding: when the channel is non-blocking; if the channel's output buffer gets full before all the data in the byte buffer (and the data pulled from the channel's input buffer into the byte buffer) gets written to it (to the channel's output buffer), then the loop will continue, thus effectively waiting (in a busy spin) until some room is created in the channel's output buffer (by some flush made toward the client) and then continues.

If so, this snippet is not non-blocking. It effectively waits until all the data in the byte buffer and all the data in the socket input buffer can be both written out to the socket output buffer, and for a slow client, it will be significant.

Please clarify is my understanding is correct. Thank you.

I tried to mitigate this by copying bytes only until the output buffer gets full (the write returns 0), and then continuing to the next channel to process. The idea being that after the next select, the channel will be further processed (even if no more bytes appear on the channel, I have a set of SelectionKeys with pending data to transfer, please check the github below).

The problem is that this mitigation does not fully work. (What works is that I can continue with the other channels, and then go back again and loop again until all the data is written out. This again will block, but fewer times). But if I simply do what I call in my code "pending writes" once and then loop again in the selector loop (do another select), then the data will be lost or corrupted, as it will come out in the output. And this I do not know why and is the other aim of the question.

Below is this other question: Why does data get lost/corrupted when not fully consuming input buffer before the next select?

The details: I am doing a non blocking echo-server with NIO: https://github.com/nmarasoiu/nio-echo-server and I have this issue:

It is only working when, after a select, I consume all the input buffer of each connection/channel and write all this data to the output buffer - but in this process I may block (via busy wait) if the output buffer gets full, essentially waiting for it to be flushed to the client, which is in effect blocking.

But if I am non-blocking, meaning I stop pumping bytes when the output buffer is full, which may happen before the input buffer becomes empty, (if I comment while (!pendingWrites.isEmpty()) from Processor), then the output gets scrambled - both lost data and corrupted data seems to be outputed.

This question appears to be off-topic. The users who voted to close gave this specific reason:

When asking a question about a problem caused by your code, you will get much better answers if you provide code people can use to reproduce the problem. Click this comment to find out how to provide what we need to help you.
– feeling unwelcome
Jun 29 at 14:44

Please read How do I ask a question that is answerable? before attempting to ask more questions so you will be better prepared and able to ask a question that will be well received and more importantly answerable.
– feeling unwelcome
Jun 29 at 14:44

1 Answer
1

Having a look at your code, I would first get it to work in single threaded mode. (Non-blocking I/O is actually intended to maximise performance on a single thread), and only after you get it robust introduce worker threads to do the business logic off the main thread.

The easiest way to get this done properly is to create an internal queue per connected client. It could be attached to the SelectionKey so that you can retrieve it easily. When you need to write to the connected client, you do the following:

SelectionKey

The select loop will then give you the client's SelectionKey with OP_WRITE when the system is ready to send (the underlying OS TCP buffers have free space). This normally happens immediately, but under high load could take some time. At that point you get the first buffer from the queue, try to send it to the client's channel, and if you manage to send all of it, you pop it off the queue and get the next one... repeating until you either finish all the queued buffers, or you dont send all the data of one of the buffers (indicating conjestion). If you send everything, change the Interested OPS back to OP_READ only.

SelectionKey

OP_WRITE

OP_READ

When you get it to work, then start to optimise (you might want to reuse the buffer rather than having a new buffer for each message). If you offload the 'worker' logic to another thread, make sure you keep the OP_WRITE and OP_READ logic in the same thread, to avoid race conditions or thread concurrency issues. You won't need multi-threading with workers until you start doing some serious business logic which involves blocking.

OP_WRITE

OP_READ

Re to your first paragraph, in fact the Processor class is an event loop Runnable that will run on a single thread and process multiple connections sequentially as events appear on them. Since we have multiple cores, then multiple event loops are useful to maximize throughput. This is why I start two processor threads. And I have an acceptor thread. 3 threads in total, unrelated to the number of connections that they process.
– NicuMarasoiu
Jun 29 at 15:52

Just be careful you are not over complicating things before getting them to work. NGINX has been single threaded for ages (and outperforming Apache). It is only relatively recently that they added the worker thread pool option. I would stick to the reactor pattern and have one event loop, events are really fast to process. Use the multiple cores to do the 'work', i.e. the business logic, which could potentially block and thus slow down the event loop. If you want to have a separate acceptor thread it is fine, but unlikely to be really offloading anything. It will be mostly idle waiting.
– jbx
Jun 29 at 16:22

On nginx.com/blog/… it is written " a small number of worker processes". My experience with Vert.x is also 1 event loop per 1 physical thread (that is 2 per core).
– NicuMarasoiu
Jun 29 at 16:49

What I am just saying is... get the simple model to work first, solving all your ordering issues cleanly and clearly without any doubts of what is happening or race conditions. Then worry about scaling up further afterwards. Trying to over optimise before getting the basic working will just make things more confusing for you to debug. Re NGINX, each process is practically standalone, so in reality you can consider it as a single threaded event driven application.
– jbx
Jun 29 at 18:50