https://blog.insiderattack.net/a-visual-guide-to-nodejs-streams-9d2d594a9bf5
Background Image Courtesy: Photo by Joshua Sortino on Unsplash
Imagine you have a pile of bricks somewhere. And you want to build a wall with those bricks here. Let’s say you have a friend to help move the bricks. To start building, you now have two options. You can either wait until your friend brings the whole pile of bricks to you, or you can start building as soon as you have a few bricks to start with, while your friend keeps bringing more bricks.
Which would be efficient? Clearly, it will be efficient to start building as soon as you have a few bricks to start with. This is a classic example of how a ‘stream’ (in this case a stream of bricks) can improve the efficiency of a process. Another common example you might be very familiar with would be streaming a movie instead downloading the whole movie first and and watch it.
In computer science, a stream is a sequence of data elements made available over time. A stream can be thought of as items on a conveyor belt being processed one at a time rather than in large batches.
In NodeJS, stream module provides the capability to work with streams. Even if you haven’t used stream module explicitly there are a lots of underlying functionality in NodeJS applications which use streams. “Streams” is an easy concept, but it may sound very complex if you are unfamiliar with it. Therefore, I thought of describing a few key concepts in NodeJS streams in visuals so that it can be easily understood.
Information is like a liquid. It flows from one place to another place as a stream of bits. For example, this happens when two peers talking to each other via a network, or even when your application is communicating with the disk or a peripheral device. When such an I/O operation happens, the information is read from a device and is flowed towards an application, or vice versa.
However, it’s possible that one end of the above transaction is slower than the other for various reasons. Therefore, some of the data might need to be “buffered” in-between, while the receiver-end is ready to accept more data.
Have a look at the following picture where two faucets of different sizes are connected via a tank. The rate water flows from upstream is higher than the rate downstream can consume. Therefore, the tank has to temporarily store (“buffer”) the excess water while the downstream slowly consumes.
This is the fundamental idea of streams in NodeJS. stream module provides functionality to implement this behaviour when working with streaming data. There are two basic types of streams provided by NodeJS.
They are,
However, there are two additional types of streams which are hybrids of Readable and Writable streams, and serve special purposes.
Let’s get into more details, and try to visualise each of these.