Jake Wharton

Forcing bytes downward in Okio

06 September 2016

Okio’s BufferedSink is a high-level abstraction for writing binary and character data as bytes. Its design stems from frustrations with the JDK’s java.io.* and java.nio.* libraries. At Droidcon Montreal last year I gave a presentation comparing it with the former, but also showcased Okio’s concept of a segment and how it enables the library to cheaply move bytes. If you aren’t familiar with Okio I encourage you to go watch the presentation first since the rest of this post will assume at least cursory knowledge of its types.

If you never look at the source code of Okio you won’t know that this segment concept exists. It’s an implementation detail for performance that remains completely opaque to the consumer of the library. That is, except for one* notable exception: the emitCompleteSegments() method on BufferedSink.

This method is part of a family of three methods which force buffered bytes to be moved to the underlying Sink. Their difference is subtle, but understanding that difference ensures correctness and can make or break throughput.

First let’s understand the difference in behavior and then look at some use cases for each.


flush()

Flush is a common concept in stream APIs and its semantics remain unchanged in Okio. Calls to this method cause all buffered bytes to be moved to the underlying Sink and then that Sink is also instructed to flush itself. When calls to flush() return, you are guaranteed that all bytes have been sent all the way to the destination Sink.

When multiple levels of buffering are in use, a call to flush() will clear the buffers at every level. In Okio multiple levels of buffering are so cheap that it’s practically free. A flush just amounts to each level moving its segments down to the next level of the chain.

With java.io.* streams, however, multiple levels of buffering require each level to allocate and manage its own byte[]. This means that a flush operation will result in each level doing an arraycopy() of its data down to the next level (which also might have required buffer expansion).

Calls to close() on stream types typically behave similar to flush() in that they write all buffered bytes to the underlying stream before also instructing it to close.

emit()

Emitting bytes is very similar to flushing except that it is not a recursive operation. Calls to this method cause all buffered bytes to be moved to the underlying Sink. Unlike flush(), however, that Sink is not told to do any other operations.

Because buffering is so inexpensive with Okio, it’s not uncommon to accept a Sink for API flexibility and immediately wrap it in a BufferedSink for the implementation’s convenience. It’s important to not leave any buffered bytes unwritten to the original Sink which is what calls to emit() will ensure.

emit() is a nice alternative to flush() since it allows you to use the more useful BufferedSink type without the concern of needlessly causing bytes to be sent all the way down the chain every time you’re finished with the abstraction.

emitCompleteSegments()

If you understand the behavior of emit() and you understood the concept of segments then the behavior of this method should be straightforward. Calls to this method cause only the bytes which are part of complete segments to be moved to the underlying Sink. If you haven’t buffered enough bytes to create a complete segment this method will actually do nothing!

Remember, segments are an implementation detail of Okio and as such so are their sizes. So why does the public API expose their concept in this method?

The reason this method exists to ensure that your code is actually not buffering too many bytes. When sending large amounts of data over a long period of time through a BufferedSink, it can be beneficial to occasionally write parts of the data to the underlying Sink. This ensures that the destination isn’t overwhelmed by a single gigantic write. Instead, the Sink can incrementally process bytes as they’re available and optionally send back signals to the producer (either with an exception or out-of-band notifications like HTTP/2’s flow control).


Now that we know the difference in behavior of these three methods, let’s look at some use cases of when you would want to use each.

Writing messages to a WebSocket

WebSockets are long-lived connections to a server over which string or binary messages are constantly streamed in both directions. The frequency of these messages can be extremely rapid or quite sparse. An API for sending messages on a WebSocket class would look something like this:

public void sendMessage(String content);
public void sendMessage(byte[] content);

This WebSocket type would wrap a socket stream with a BufferedSink for sending messages. Because we don’t know when the next message will come from the application, the implementation of sendMessage needs to call flush() before returning to optimize for latency. This ensures all of the data from each message will be sent down through the socket.

If you were to use emitCompleteSegments() part of the message would almost always be left in the buffer. Using emit() would only work if there’s no intermediate buffering which is hard to guarantee. This is why flush() is the only appropriate operation for this example.

Encoding a video to a file

Video encoding is a CPU-bound, memory-intensive process which generates data at a fairly consistent rate. Writing this data to a file as its being encoded keeps the buffer size small and ensures that the slower disk drive can keep up.

At regular intervals the implementation writing encoded data to a BufferedSink should call emitCompleteSegments() to allow large portions of the buffer to get moved to the underlying Sink and start trickling down the chain. The reason that emitCompleteSegments() is preferred here over emit() is that more data will be coming into buffer. Sending a partially-completed segment would be wasteful since it has empty bytes that can still be used for the incoming data.

It’s important to note again that emitCompleteSegments() only writes to the underlying Sink and not all the way down the chain (i.e., it’s not flushCompleteSegments()). This means that if there is an intermediate buffer which isn’t monitoring its size and occasionally calling this method you will end up buffering the whole video.

When the video is done encoding and no more bytes will be written a call to emit() (with the same caveats as the last paragraph) or flush() should happen so that the final bytes are not left in the buffer.

Serializing an object to JSON

If we wanted to create a library that took an object and serialized it to JSON we would probably give it a method signature like this:

public void toJson(Object o, Sink sink)

Because Sink offers no convenience on its own, the implementation would buffer it into an BufferedSink for access to high-level APIs.

Once the implementation was finished writing the JSON representation to the BufferedSink it needs to return. In order to not leave any bytes in the buffer it needs to call emit(). This writes all of the buffered bytes to the underlying Sink, but not any further. Whether or not the Sink should be flushed all the way down is left as a decision for the caller. This affords more control so that if the caller wants to serialize multiple objects and flush them all at once they are able to do so.

Astute readers might be aware of Moshi–a small JSON serialization library built with Okio. Unlike our hypothetical toJson method above, Moshi’s method actually requires a BufferedSink directly. The reason for this is completely unrelated to flushing or emitting data, but rather for symmetry with the fromJson method which requires a BufferedSource.


So flush(), emit(), and emitCompleteSegments() each instruct a BufferedSink to move data to the underlying Sink in slightly different ways. Understanding that difference ensures that your bytes do not get lost inside of intermediate buffers but also that you move the minimal amount of bytes for your needs.

 

 

 

* Technically there is two exceptions. Buffer has a completeSegmentByteCount() method which returns the number of bytes that would be moved by a call to emitCompleteSegments().

— Jake Wharton