Node.js Streams

What is Streams in Node.js?

A stream processes data piece by piece instead of loading it all into memory. Node has four kinds — readable, writable, duplex and transform — and they are how large files, uploads and network data are handled efficiently.

Why Streams matters

Reading a two-gigabyte file with readFile needs two gigabytes of memory and fails under load. A stream processes it in small chunks with near-constant memory use.

Streams example

import { createReadStream, createWriteStream } from 'node:fs';
import { pipeline } from 'node:stream/promises';
import { createGzip } from 'node:zlib';

await pipeline(
  createReadStream('big.csv'),
  createGzip(),
  createWriteStream('big.csv.gz')
);
console.log('Compressed without loading the file into memory');

How this works

pipeline connects the streams, forwards errors, and cleans up every stream if any stage fails — which is why it is preferred over chaining .pipe() calls manually.

Reading line by line

import { createInterface } from 'node:readline';

const rl = createInterface({ input: createReadStream('jobs.csv') });
for await (const line of rl) {
  process(line);   // one line at a time, whatever the file size
}

Key points to remember

• Always use pipeline rather than manual .pipe() — it propagates errors and cleans up.
• Backpressure is handled automatically when you use pipe or pipeline.
• HTTP requests and responses are already streams.

Stream types

Common mistakes with Streams

• Chained .pipe() calls that leak file descriptors when a middle stage errors.
• Buffering an entire upload in memory before writing it to disk.