We deserve a greater streams API for JavaScript

Dealing with information in streams is prime to how we construct functions. To make streaming work in all places, the WHATWG Streams Customary (informally referred to as “Web streams”) was designed to determine a typical API to work throughout browsers and servers. It shipped in browsers, was adopted by Cloudflare Staff, Node.js, Deno, and Bun, and have become the inspiration for APIs like fetch(). It is a vital endeavor, and the individuals who designed it have been fixing laborious issues with the constraints and instruments they’d on the time.

However after years of constructing on Net streams – implementing them in each Node.js and Cloudflare Staff, debugging manufacturing points for patrons and runtimes, and serving to builders work by way of far too many widespread pitfalls – I’ve come to consider that the usual API has basic usability and efficiency points that can’t be mounted simply with incremental enhancements alone. The issues aren’t bugs; they’re penalties of design selections which will have made sense a decade in the past, however do not align with how JavaScript builders write code at this time.

This put up explores among the basic points I see with Net streams and presents another strategy constructed round JavaScript language primitives that display one thing higher is feasible. 

In benchmarks, this various can run wherever between 2x to 120x sooner than Net streams in each runtime I’ve examined it on (together with Cloudflare Staff, Node.js, Deno, Bun, and each main browser). The enhancements should not on account of intelligent optimizations, however basically totally different design decisions that extra successfully leverage fashionable JavaScript language options. I am not right here to disparage the work that got here earlier than; I am right here to begin a dialog about what can doubtlessly come subsequent.

The Streams Customary was developed between 2014 and 2016 with an bold purpose to offer “APIs for creating, composing, and consuming streams of data that map efficiently to low-level I/O primitives.” Earlier than Net streams, the online platform had no commonplace strategy to work with streaming information.

Node.js already had its personal streaming API on the time that was ported to additionally work in browsers, however WHATWG selected to not use it as a place to begin provided that it’s chartered to solely contemplate the wants of Net browsers. Server-side runtimes solely adopted Net streams later, after Cloudflare Staff and Deno every emerged with first-class Net streams help and cross-runtime compatibility turned a precedence.

The design of Net streams predates async iteration in JavaScript. The for await...of syntax did not land till ES2018, two years after the Streams Customary was initially finalized. This timing meant the API could not initially leverage what would ultimately grow to be the idiomatic strategy to eat asynchronous sequences in JavaScript. As an alternative, the spec launched its personal reader/author acquisition mannequin, and that call rippled by way of each side of the API.

The commonest job with streams is studying them to completion. This is what that appears like with Net streams:

// First, we purchase a reader that offers an unique lock
// on the stream...
const reader = stream.getReader();
const chunks = [];
attempt {
  // Second, we repeatedly name learn and await on the returned
  // promise to both yield a bit of knowledge or point out we're
  // performed.
  whereas (true) {
    const { worth, performed } = await reader.learn();
    if (performed) break;
    chunks.push(worth);
  }
} lastly {
  // Lastly, we launch the lock on the stream
  reader.releaseLock();
}

You would possibly assume this sample is inherent to streaming. It is not. The reader acquisition, the lock administration, and the { worth, performed } protocol are all simply design decisions, not necessities. They’re artifacts of how and when the Net streams spec was written. Async iteration exists exactly to deal with sequences that arrive over time, however async iteration didn’t but exist when the streams specification was written. The complexity right here is pure API overhead, not basic necessity.

Think about the choice strategy now that Net streams do help for await...of:

const chunks = [];
for await (const chunk of stream) {
  chunks.push(chunk);
}

That is higher in that there’s far much less boilerplate, nevertheless it would not clear up every part. Async iteration was retrofitted onto an API that wasn’t designed for it, and it reveals. Options like BYOB (carry your individual buffer) reads aren’t accessible by way of iteration. The underlying complexity of readers, locks, and controllers are nonetheless there, simply hidden. When one thing does go unsuitable, or when extra options of the API are wanted, builders discover themselves again within the weeds of the unique API, attempting to know why their stream is “locked” or why releaseLock() did not do what they anticipated or searching down bottlenecks in code they do not management.

Net streams use a locking mannequin to stop a number of shoppers from interleaving reads. If you name getReader(), the stream turns into locked. Whereas locked, nothing else can learn from the stream immediately, pipe it, and even cancel it – solely the code that’s really holding the reader can.

This sounds affordable till you see how simply it goes unsuitable:

async perform peekFirstChunk(stream) {
  const reader = stream.getReader();
  const { worth } = await reader.learn();
  // Oops — forgot to name reader.releaseLock()
  // And the reader is not out there once we return
  return worth;
}

const first = await peekFirstChunk(stream);
// TypeError: Can't acquire lock — stream is completely locked
for await (const chunk of stream) { /* by no means runs */ }

Forgetting releaseLock() completely breaks the stream. The locked property tells you {that a} stream is locked, however not why, by whom, or whether or not the lock is even nonetheless usable. Piping internally acquires locks, making streams unusable throughout pipe operations in ways in which aren’t apparent.

The semantics round releasing locks with pending reads have been additionally unclear for years. In the event you known as learn() however did not await it, then known as releaseLock(), what occurred? The spec was lately clarified to cancel pending reads on lock launch – however implementations diverse, and code that relied on the earlier unspecified conduct can break.

That mentioned, it is essential to acknowledge that locking in itself is just not dangerous. It does, in truth, serve an essential objective to make sure that functions correctly and orderly eat or produce information. The important thing problem is with the unique guide implementation of it utilizing APIs like getReader() and releaseLock(). With the arrival of computerized lock and reader administration with async iterables, coping with locks from the customers perspective turned loads simpler.

For implementers, the locking mannequin provides a good quantity of non-trivial inner bookkeeping. Each operation should verify lock state, readers should be tracked, and the interaction between locks, cancellation, and error states creates a matrix of edge instances that should all be dealt with appropriately.

BYOB (carry your individual buffer) reads have been designed to let builders reuse reminiscence buffers when studying from streams, an essential optimization meant for high-throughput situations. The thought is sound: as a substitute of allocating new buffers for every chunk, you present your individual buffer and the stream fills it.

In observe, (and sure, there are all the time exceptions to be discovered) BYOB isn’t used to any measurable profit. The API is considerably extra advanced than default reads, requiring a separate reader kind (ReadableStreamBYOBReader) and different specialised lessons (e.g. ReadableStreamBYOBRequest), cautious buffer lifecycle administration, and understanding of ArrayBuffer detachment semantics. If you cross a buffer to a BYOB learn, the buffer turns into indifferent – transferred to the stream – and also you get again a unique view over doubtlessly totally different reminiscence. This transfer-based mannequin is error-prone and complicated:

const reader = stream.getReader({ mode: 'byob' });
const buffer = new ArrayBuffer(1024);
let view = new Uint8Array(buffer);

const outcome = await reader.learn(view);
// 'view' ought to now be indifferent and unusable
// (it is not all the time in each impl)
// outcome.worth is a NEW view, presumably over totally different reminiscence
view = outcome.worth; // Should reassign

BYOB additionally cannot be used with async iteration or TransformStreams, so builders who need zero-copy reads are compelled again into the guide reader loop.

For implementers, BYOB provides vital complexity. The stream should observe pending BYOB requests, deal with partial fills, handle buffer detachment appropriately, and coordinate between the BYOB reader and the underlying supply. The Net Platform Checks for readable byte streams embrace devoted check recordsdata only for BYOB edge instances: indifferent buffers, dangerous views, response-after-enqueue ordering, and extra.

BYOB finally ends up being advanced for each customers and implementers, but sees little adoption in observe. Most builders follow default reads and settle for the allocation overhead.

Most userland implementations of customized ReadableStream situations don’t usually hassle with all of the ceremony required to appropriately implement each default and BYOB learn help in a single stream – and for good cause. It is troublesome to get proper and more often than not consuming code is often going to fallback on the default learn path. The instance under reveals what a “correct” implementation would want to do. It is massive, advanced, and error inclined, and never a degree of complexity that the standard developer actually needs to must take care of:

new ReadableStream({
    kind: 'bytes',
    
    async pull(controller: ReadableByteStreamController) {      
      if (offset >= totalBytes) {
        controller.shut();
        return;
      }
      
      // Verify for BYOB request FIRST
      const byobRequest = controller.byobRequest;
      
      if (byobRequest) {
        // === BYOB PATH ===
        // Shopper offered a buffer - we MUST fill it (or a part of it)
        const view = byobRequest.view!;
        const bytesAvailable = totalBytes - offset;
        const bytesToWrite = Math.min(view.byteLength, bytesAvailable);
        
        // Create a view into the patron's buffer and fill it
        // not crucial however safer when bytesToWrite != view.byteLength
        const dest = new Uint8Array(
          view.buffer,
          view.byteOffset,
          bytesToWrite
        );
        
        // Fill with sequential bytes (our "data source")
        // Might be any factor right here that writes into the view
        for (let i = 0; i < bytesToWrite; i++) {
          dest[i] = (offset + i) & 0xFF;
        }
        
        offset += bytesToWrite;
        
        // Sign what number of bytes we wrote
        byobRequest.reply(bytesToWrite);
        
      } else {
        // === DEFAULT READER PATH ===
        // No BYOB request - allocate and enqueue a bit
        const bytesAvailable = totalBytes - offset;
        const chunkSize = Math.min(1024, bytesAvailable);
        
        const chunk = new Uint8Array(chunkSize);
        for (let i = 0; i < chunkSize; i++) {
          chunk[i] = (offset + i) & 0xFF;
        }
        
        offset += chunkSize;
        controller.enqueue(chunk);
      }
    },
    
    cancel(cause) {
      console.log('Stream canceled:', cause);
    }
  });

When a bunch runtime supplies a byte-oriented ReadableStream from the runtime itself, as an illustration, because the physique of a fetch Response, it’s usually far simpler for the runtime itself to offer an optimized implementation of BYOB reads, however these nonetheless must be able to dealing with each default and BYOB studying patterns and that requirement brings with it a good quantity of complexity.

Backpressure – the flexibility for a gradual client to sign a quick producer to decelerate – is a first-class idea in Net streams. In principle. In observe, the mannequin has some critical flaws.

The first sign is desiredSize on the controller. It may be optimistic (needs information), zero (at capability), unfavourable (over capability), or null (closed). Producers are purported to verify this worth and cease enqueueing when it is not optimistic. However there’s nothing imposing this: controller.enqueue() all the time succeeds, even when desiredSize is deeply unfavourable.

new ReadableStream({
  begin(controller) {
    // Nothing stops you from doing this
    whereas (true) {
      controller.enqueue(generateData()); // desiredSize: -999999
    }
  }
});

Stream implementations can and do ignore backpressure; and a few spec-defined options explicitly break backpressure. tee(), as an illustration, creates two branches from a single stream. If one department reads sooner than the opposite, information accumulates in an inner buffer with no restrict. A quick client could cause unbounded reminiscence development whereas the gradual client catches up, and there is no strategy to configure this or choose out past canceling the slower department.

Net streams do present clear mechanisms for tuning backpressure conduct within the type of the highWaterMark choice and customizable dimension calculations, however these are simply as straightforward to disregard as desiredSize, and lots of functions merely fail to concentrate to them.

The identical points exist on the WritableStream facet. A WritableStream has a highWaterMark and desiredSize. There’s a author.prepared promise that producers of knowledge are supposed to concentrate however usually do not.

const writable = getWritableStreamSomehow();
const author = writable.getWriter();

// Producers are supposed to attend for the author.prepared
// It's a promise that, when resolves, signifies that
// the writables inner backpressure is cleared and
// it's okay to put in writing extra information
await author.prepared;
await author.write(...);

For implementers, backpressure provides complexity with out offering ensures. The equipment to trace queue sizes, compute desiredSize, and invoke pull() on the proper instances should all be applied appropriately. Nonetheless, since these indicators are advisory, all that work would not really forestall the issues backpressure is meant to resolve.

The Net streams spec requires promise creation at quite a few factors, usually in scorching paths and sometimes invisible to customers. Every learn() name would not simply return a promise; internally, the implementation creates extra guarantees for queue administration, pull() coordination, and backpressure signaling.

This overhead is remitted by the spec’s reliance on guarantees for buffer administration, completion, and backpressure indicators. Whereas a few of it’s implementation-specific, a lot of it’s unavoidable in the event you’re following the spec as written. For top-frequency streaming – video frames, community packets, real-time information – this overhead is important.

The issue compounds in pipelines. Every TransformStream provides one other layer of promise equipment between supply and sink. The spec would not outline synchronous quick paths, so even when information is out there instantly, the promise equipment nonetheless runs.

For implementers, this promise-heavy design constrains optimization alternatives. The spec mandates particular promise decision ordering, making it troublesome to batch operations or skip pointless async boundaries with out risking delicate compliance failures. There are a lot of hidden inner optimizations that implementers do make however these might be sophisticated and troublesome to get proper.

Whereas I used to be penning this weblog put up, Vercel’s Malte Ubl revealed their very own weblog put up describing some analysis work Vercel has been doing round enhancing the efficiency of Node.js’ Net streams implementation. In that put up they talk about the identical basic efficiency optimization downside that each implementation of Net streams face:

“Or consider pipeTo(). Each chunk passes through a full Promise chain: read, write, check backpressure, repeat. An {value, done} result object is allocated per read. Error propagation creates additional Promise branches.

None of this is wrong. These guarantees matter in the browser where streams cross security boundaries, where cancellation semantics need to be airtight, where you do not control both ends of a pipe. But on the server, when you are piping React Server Components through three transforms at 1KB chunks, the cost adds up.

We benchmarked native WebStream pipeThrough at 630 MB/s for 1KB chunks. Node.js pipeline() with the same passthrough transform: ~7,900 MB/s. That is a 12x gap, and the difference is almost entirely Promise and object allocation overhead.”
– Malte Ubl, https://vercel.com/blog/we-ralph-wiggumed-webstreams-to-make-them-10x-faster

As a part of their analysis, they’ve put collectively a set of proposed enhancements for Node.js’ Net streams implementation that may remove guarantees in sure code paths which might yield a major efficiency increase as much as 10x sooner, which solely goes to show the purpose: guarantees, whereas helpful, add vital overhead. As one of many core maintainers of Node.js, I’m trying ahead to serving to Malte and the oldsters at Vercel get their proposed enhancements landed!

In a latest replace made to Cloudflare Staff, I made comparable sorts of modifications to an inner information pipeline that diminished the variety of JavaScript guarantees created in sure software situations by as much as 200x. The result’s a number of orders of magnitude enchancment in efficiency in these functions.

When fetch() returns a response, the physique is a ReadableStream. In the event you solely verify the standing and do not eat or cancel the physique, what occurs? The reply varies by implementation, however a typical final result is useful resource leakage.

async perform checkEndpoint(url) {
  const response = await fetch(url);
  return response.okay; // Physique isn't consumed or cancelled
}

// In a loop, this may exhaust connection swimming pools
for (const url of urls) {
  await checkEndpoint(url);
}

This sample has induced connection pool exhaustion in Node.js functions utilizing undici (the fetch() implementation constructed into Node.js), and comparable points have appeared in different runtimes. The stream holds a reference to the underlying connection, and with out specific consumption or cancellation, the connection might linger till rubbish assortment – which can not occur quickly sufficient beneath load.

The issue is compounded by APIs that implicitly create stream branches. Request.clone() and Response.clone() carry out implicit tee() operations on the physique stream – a element that is straightforward to overlook. Code that clones a request for logging or retry logic might unknowingly create branched streams that want impartial consumption, multiplying the useful resource administration burden.

Now, to make certain, a lot of these points are implementation bugs. The connection leak was positively one thing that undici wanted to repair in its personal implementation, however the complexity of the specification doesn’t make coping with a lot of these points straightforward.

“Cloning streams in Node.js’s fetch() implementation is harder than it looks. When you clone a request or response body, you’re calling tee() – which splits a single stream into two branches that both need to be consumed. If one consumer reads faster than the other, data buffers unbounded in memory waiting for the slow branch. If you don’t properly consume both branches, the underlying connection leaks. The coordination required between two readers sharing one source makes it easy to accidentally break the original request or exhaust connection pools. It’s a simple API call with complex underlying mechanics that are difficult to get right.” – Matteo Collina, Ph.D. – Platformatic Co-Founder & CTO, Node.js Technical Steering Committee Chair

tee() splits a stream into two branches. It appears simple, however the implementation requires buffering: if one department is learn sooner than the opposite, the info should be held someplace till the slower department catches up.

const [forHash, forStorage] = response.physique.tee();

// Hash computation is quick
const hash = await computeHash(forHash);

// Storage write is gradual — in the meantime, your entire stream
// could also be buffered in reminiscence ready for this department
await writeToStorage(forStorage);

The spec doesn’t mandate buffer limits for tee(). And to be honest, the spec permits implementations to implement the precise inner mechanisms for tee()and different APIs in any means they see match as long as the observable normative necessities of the specification are met. But when an implementation chooses to implement tee() within the particular means described by the streams specification, then tee() will include a built-in reminiscence administration difficulty that’s troublesome to work round.

Implementations have needed to develop their very own methods for coping with this. Firefox initially used a linked-list strategy that led to O(n) reminiscence development proportional to the consumption fee distinction. In Cloudflare Staff, we opted to implement a shared buffer mannequin the place backpressure is signaled by the slowest client slightly than the quickest.

TransformStream creates a readable/writable pair with processing logic in between. The rework() perform executes on write, not on learn. Processing of the rework occurs eagerly as information arrives, no matter whether or not any client is prepared. This causes pointless work when shoppers are gradual, and the backpressure signaling between the 2 sides has gaps that may trigger unbounded buffering beneath load. The expectation within the spec is that the producer of the info being reworked is taking note of the author.prepared sign on the writable facet of the rework however very often producers simply merely ignore it.

If the rework’s rework() operation is synchronous and all the time enqueues output instantly, it by no means indicators backpressure again to the writable facet even when the downstream client is gradual. It is a consequence of the spec design that many builders fully overlook. In browsers, the place there’s solely a single person and usually solely a small variety of stream pipelines energetic at any given time, this kind of foot gun is usually of no consequence, nevertheless it has a serious influence on server-side or edge efficiency in runtimes that serve 1000’s of concurrent requests.

const fastTransform = new TransformStream({
  rework(chunk, controller) {
    // Synchronously enqueue — this by no means applies backpressure
    // Even when the readable facet's buffer is full, this succeeds
    controller.enqueue(processChunk(chunk));
  }
});

// Pipe a quick supply by way of the rework to a gradual sink
fastSource
  .pipeThrough(fastTransform)
  .pipeTo(slowSink);  // Buffer grows with out sure

What TransformStreams are purported to do is verify for backpressure on the controller and use guarantees to speak that again to the author:

const fastTransform = new TransformStream({
  async rework(chunk, controller) {
    if (controller.desiredSize <= 0) {
      // Wait on the backpressure to clear by some means
    }

    controller.enqueue(processChunk(chunk));
  }
});

An issue right here, nevertheless, is that the TransformStreamDefaultController doesn’t have a prepared promise mechanism like Writers do; so the TransformStream implementation would want to implement a polling mechanism to periodically verify when controller.desiredSize turns into optimistic once more.

The issue will get worse in pipelines. If you chain a number of transforms – say, parse, rework, then serialize – every TransformStream has its personal inner readable and writable buffers. If implementers comply with the spec strictly, information cascades by way of these buffers in a push-oriented style: the supply pushes to rework A, which pushes to rework B, which pushes to rework C, every accumulating information in intermediate buffers earlier than the ultimate client has even began pulling. With three transforms, you may have six inner buffers filling up concurrently.

Builders utilizing the streams API are anticipated to recollect to make use of choices like highWaterMark when creating their sources, transforms, and writable locations however usually they both neglect or just select to disregard it.

supply
  .pipeThrough(parse)      // buffers filling...
  .pipeThrough(rework)  // extra buffers filling...
  .pipeThrough(serialize)  // much more buffers...
  .pipeTo(vacation spot);    // client hasn't began but

Implementations have discovered methods to optimize rework pipelines by collapsing id transforms, short-circuiting non-observable paths, deferring buffer allocation, or falling again to native code that doesn’t run JavaScript in any respect. Deno, Bun, and Cloudflare Staff have all efficiently applied “native path” optimizations that may assist remove a lot of the overhead, and Vercel’s latest fast-webstreams analysis is engaged on comparable optimizations for Node.js. However the optimizations themselves add vital complexity and nonetheless cannot totally escape the inherently push-oriented mannequin that TransformStream makes use of.

Streaming server-side rendering (SSR) is a very painful case. A typical SSR stream would possibly render 1000’s of small HTML fragments, every passing by way of the streams equipment:

// Every part enqueues a small chunk
perform renderComponent(controller) {
  controller.enqueue(encoder.encode(`
${content material}
`));
}

// Tons of of parts = a whole lot of enqueue calls
// Every one triggers promise equipment internally
for (const part of parts) {
  renderComponent(controller);  // Guarantees created, objects allotted
}

Each fragment means guarantees created for learn() calls, guarantees for backpressure coordination, intermediate buffer allocations, and { worth, performed } outcome objects – most of which grow to be rubbish nearly instantly.

Underneath load, this creates GC strain that may devastate throughput. The JavaScript engine spends vital time gathering short-lived objects as a substitute of doing helpful work. Latency turns into unpredictable as GC pauses interrupt request dealing with. I’ve seen SSR workloads the place rubbish assortment accounts for a considerable portion (as much as and past 50%) of whole CPU time per request. That is time that might be spent really rendering content material.

The irony is that streaming SSR is meant to enhance efficiency by sending content material incrementally. However the overhead of the streams equipment can negate these good points, particularly for pages with many small parts. Builders generally discover that buffering your entire response is definitely sooner than streaming by way of Net streams, defeating the aim completely.

To attain usable efficiency, each main runtime has resorted to non-standard inner optimizations for Net streams. Node.js, Deno, Bun, and Cloudflare Staff have all developed their very own workarounds. That is notably true for streams wired as much as system-level I/O, the place a lot of the equipment is non-observable and might be short-circuited.

Discovering these optimization alternatives can itself be a major endeavor. It requires end-to-end understanding of the spec to determine which behaviors are observable and which might safely be elided. Even then, whether or not a given optimization is definitely spec-compliant is usually unclear. Implementers should make judgment calls about which semantics they will chill out with out breaking compatibility. This places huge strain on runtime groups to grow to be spec consultants simply to attain acceptable efficiency.

These optimizations are troublesome to implement, often error-prone, and result in inconsistent conduct throughout runtimes. Bun’s “Direct Streams” optimization takes a intentionally and observably non-standard strategy, bypassing a lot of the spec’s equipment completely. Cloudflare Staff’ IdentityTransformStream supplies a fast-path for pass-through transforms however is Staff-specific and implements behaviors that aren’t commonplace for a TransformStream. Every runtime has its personal set of tips and the pure tendency is towards non-standard options, as a result of that is usually the one strategy to make issues quick.

This fragmentation hurts portability. Code that performs properly on one runtime might behave in a different way (or poorly) on one other, although it is utilizing “standard” APIs. The complexity burden on runtime implementers is substantial, and the delicate behavioral variations create friction for builders attempting to put in writing cross-runtime code, notably these sustaining frameworks that should have the ability to run effectively throughout many runtime environments.

It’s also vital to emphasise that many optimizations are solely potential in elements of the spec which are unobservable to person code. The choice, like Bun “Direct Streams”, is to deliberately diverge from the spec-defined observable behaviors. This implies optimizations usually really feel “incomplete”. They work in some situations however not in others, in some runtimes however not others, and so on. Each such case provides to the general unsustainable complexity of the Net streams strategy which is why most runtime implementers not often put vital effort into additional enhancements to their streams implementations as soon as the conformance checks are passing.

Implementers should not want to leap by way of these hoops. When you end up needing to chill out or bypass spec semantics simply to attain affordable efficiency, that is an indication one thing is unsuitable with the spec itself. A well-designed streaming API ought to be environment friendly by default, not require every runtime to invent its personal escape hatches.

A fancy spec creates advanced edge instances. The Net Platform Checks for streams span over 70 check recordsdata, and whereas complete testing is an efficient factor, what’s telling is what must be examined.

Think about among the extra obscure checks that implementations should cross:

• Prototype air pollution protection: One check patches Object.prototype.then to intercept promise resolutions, then verifies that pipeTo() and tee() operations do not leak inner values by way of the prototype chain. This checks a safety property that solely exists as a result of the spec’s promise-heavy internals create an assault floor.

• WebAssembly reminiscence rejection: BYOB reads should explicitly reject ArrayBuffers backed by WebAssembly reminiscence, which appear like common buffers however cannot be transferred. This edge case exists due to the spec’s buffer detachment mannequin – a less complicated API would not have to deal with it.

• Crash regression for state machine conflicts: A check particularly checks that calling byobRequest.reply() after enqueue() would not crash the runtime. This sequence creates a battle within the inner state machine — the enqueue() fulfills the pending learn and will invalidate the byobRequest, however implementations should gracefully deal with the next reply() slightly than corrupting reminiscence so as to cowl the very doubtless risk that builders should not utilizing the advanced API appropriately.

These aren’t contrived situations invented by check authors in whole vacuum. They’re penalties of the spec’s design and mirror actual world bugs.

For runtime implementers, passing the WPT suite means dealing with intricate nook instances that almost all software code won’t ever encounter. The checks encode not simply the comfortable path however the full matrix of interactions between readers, writers, controllers, queues, methods, and the promise equipment that connects all of them.

A less complicated API would imply fewer ideas, fewer interactions between ideas, and fewer edge instances to get proper leading to extra confidence that implementations really behave constantly.

Net streams are advanced for customers and implementers alike. The issues with the spec aren’t bugs. They emerge from utilizing the API precisely as designed. They don’t seem to be points that may be mounted solely by way of incremental enhancements. They’re penalties of basic design decisions. To enhance issues we’d like totally different foundations.

After implementing the Net streams spec a number of instances throughout totally different runtimes and seeing the ache factors firsthand, I made a decision it was time to discover what a greater, various streaming API might appear like if designed from first ideas at this time.

What follows is a proof of idea: it is not a completed commonplace, not a production-ready library, not even essentially a concrete proposal for one thing new, however a place to begin for dialogue that demonstrates the issues with Net streams aren’t inherent to streaming itself; they’re penalties of particular design decisions that might be made in a different way. Whether or not this actual API is the precise reply is much less essential than whether or not it sparks a productive dialog about what we really need from a streaming primitive.

Earlier than diving into API design, it is value asking: what’s a stream?

At its core, a stream is only a sequence of knowledge that arrives over time. You do not have all of it without delay. You course of it incrementally because it turns into out there.

Unix pipes are maybe the purest expression of this concept:

cat entry.log | grep "error" | type | uniq -c

Knowledge flows left to proper. Every stage reads enter, does its work, writes output. There is not any pipe reader to accumulate, no controller lock to handle. If a downstream stage is gradual, upstream phases naturally decelerate as properly. Backpressure is implicit within the mannequin, not a separate mechanism to study (or ignore).

In JavaScript, the pure primitive for “a sequence of things that arrive over time” is already within the language: the async iterable. You eat it with for await...of. You cease consuming by stopping iteration.

That is the instinct the brand new API tries to protect: streams ought to really feel like iteration, as a result of that is what they’re. The complexity of Net streams – readers, writers, controllers, locks, queuing methods – obscures this basic simplicity. A greater API ought to make the easy case easy and solely add complexity the place it is genuinely wanted.

I constructed the proof-of-concept various round a unique set of ideas.

No customized ReadableStream class with hidden inner state. A readable stream is simply an AsyncIterable. You eat it with for await...of. No readers to accumulate, no locks to handle.

Transforms do not execute till the patron pulls. There is not any keen analysis, no hidden buffering. Knowledge flows on-demand from supply, by way of transforms, to the patron. In the event you cease iterating, processing stops.

Backpressure is strict by default. When a buffer is full, writes reject slightly than silently accumulating. You possibly can configure various insurance policies – block till area is out there, drop oldest, drop latest – however it’s important to select explicitly. No extra silent reminiscence development.

As an alternative of yielding one chunk per iteration, streams yield Uint8Array[]: arrays of chunks. This amortizes the async overhead throughout a number of chunks, lowering promise creation and microtask latency in scorching paths.

The API offers completely with bytes (Uint8Array). Strings are UTF-8 encoded robotically. There is not any “value stream” vs “byte stream” dichotomy. If you wish to stream arbitrary JavaScript values, use async iterables immediately. Whereas the API makes use of Uint8Array, it treats chunks as opaque. There isn’t a partial consumption, no BYOB patterns, no byte-level operations throughout the streaming equipment itself. Chunks go in, chunks come out, unchanged until a rework explicitly modifies them.

The API acknowledges that synchronous information sources are each vital and customary. The applying shouldn’t be compelled to all the time settle for the efficiency price of asynchronous scheduling just because that is the one choice offered. On the similar time, mixing sync and async processing might be harmful. Synchronous paths ought to all the time be an choice and will all the time be specific.

In Net streams, making a easy producer/client pair requires TransformStream, guide encoding, and cautious lock administration:

const { readable, writable } = new TransformStream();
const enc = new TextEncoder();
const author = writable.getWriter();
await author.write(enc.encode("Hello, World!"));
await author.shut();
author.releaseLock();

const dec = new TextDecoder();
let textual content = '';
for await (const chunk of readable) {
  textual content += dec.decode(chunk, { stream: true });
}
textual content += dec.decode();

Even this comparatively clear model requires: a TransformStream, guide TextEncoder and TextDecoder, and specific lock launch.

This is the equal with the brand new API:

import { Stream } from 'new-streams';

// Create a push stream
const { author, readable } = Stream.push();

// Write information — backpressure is enforced
await author.write("Hello, World!");
await author.finish();

// Eat as textual content
const textual content = await Stream.textual content(readable);

The readable is simply an async iterable. You possibly can cross it to any perform that expects one, together with Stream.textual content() which collects and decodes your entire stream.

The author has a easy interface: write(), writev() for batched writes, finish() to sign completion, and abort() for errors. That is primarily it.

The Author is just not a concrete class. Any object that implements write(), finish(), and abort() is usually a author making it straightforward to adapt present APIs or create specialised implementations with out subclassing. There is not any advanced UnderlyingSink protocol with begin(), write(), shut(), and abort() callbacks that should coordinate by way of a controller whose lifecycle and state are impartial of the WritableStream it’s sure to.

This is a easy in-memory author that collects all written information:

// A minimal author implementation — simply an object with strategies
perform createBufferWriter() {
  const chunks = [];
  let totalBytes = 0;
  let closed = false;

  const addChunk = (chunk) => {
    chunks.push(chunk);
    totalBytes += chunk.byteLength;
  };

  return {
    get desiredSize() { return closed ? null : 1; },

    // Async variants
    write(chunk) { addChunk(chunk); },
    writev(batch) { for (const c of batch) addChunk(c); },
    finish() { closed = true; return totalBytes; },
    abort(cause) { closed = true; chunks.size = 0; },

    // Sync variants return boolean (true = accepted)
    writeSync(chunk) { addChunk(chunk); return true; },
    writevSync(batch) { for (const c of batch) addChunk(c); return true; },
    endSync() { closed = true; return totalBytes; },
    abortSync(cause) { closed = true; chunks.size = 0; return true; },

    getChunks() { return chunks; }
  };
}

// Use it
const author = createBufferWriter();
await Stream.pipeTo(supply, author);
const allData = author.getChunks();

No base class to increase, no summary strategies to implement, no controller to coordinate with. Simply an object with the precise form.

Underneath the brand new API design, transforms shouldn’t carry out any work till the info is being consumed. It is a basic precept.

// Nothing executes till iteration begins
const output = Stream.pull(supply, compress, encrypt);

// Transforms execute as we iterate
for await (const chunks of output) {
  for (const chunk of chunks) {
    course of(chunk);
  }
}

Stream.pull() creates a lazy pipeline. The compress and encrypt transforms do not run till you begin iterating output. Every iteration pulls information by way of the pipeline on demand.

That is basically totally different from Net streams’ pipeThrough(), which begins actively pumping information from the supply to the rework as quickly as you arrange the pipe. Pull semantics imply you management when processing occurs, and stopping iteration stops processing.

Transforms might be stateless or stateful. A stateless rework is only a perform that takes chunks and returns reworked chunks:

// Stateless rework — a pure perform
// Receives chunks or null (flush sign)
const toUpperCase = (chunks) => {
  if (chunks === null) return null; // Finish of stream
  return chunks.map(chunk => {
    const str = new TextDecoder().decode(chunk);
    return new TextEncoder().encode(str.toUpperCase());
  });
};

// Use it immediately
const output = Stream.pull(supply, toUpperCase);

Stateful transforms are easy objects with member features that keep state throughout calls:

// Stateful rework — a generator that wraps the supply
perform createLineParser() {
  // Helper to concatenate Uint8Arrays
  const concat = (...arrays) => {
    const outcome = new Uint8Array(arrays.scale back((n, a) => n + a.size, 0));
    let offset = 0;
    for (const arr of arrays) { outcome.set(arr, offset); offset += arr.size; }
    return outcome;
  };

  return {
    async *rework(supply) {
      let pending = new Uint8Array(0);
      
      for await (const chunks of supply) {
        if (chunks === null) {
          // Flush: yield any remaining information
          if (pending.size > 0) yield [pending];
          proceed;
        }
        
        // Concatenate pending information with new chunks
        const mixed = concat(pending, ...chunks);
        const strains = [];
        let begin = 0;

        for (let i = 0; i < mixed.size; i++) {
          if (mixed[i] === 0x0a) { // newline
            strains.push(mixed.slice(begin, i));
            begin = i + 1;
          }
        }

        pending = mixed.slice(begin);
        if (strains.size > 0) yield strains;
      }
    }
  };
}

const output = Stream.pull(supply, createLineParser());

For transforms that want cleanup on abort, add an abort handler:

// Stateful rework with useful resource cleanup
perform createGzipCompressor() {
  // Hypothetical compression API...
  const deflate = new Deflater({ gzip: true });

  return {
    async *rework(supply) {
      for await (const chunks of supply) {
        if (chunks === null) {
          // Flush: finalize compression
          deflate.push(new Uint8Array(0), true);
          if (deflate.outcome) yield [deflate.result];
        } else {
          for (const chunk of chunks) {
            deflate.push(chunk, false);
            if (deflate.outcome) yield [deflate.result];
          }
        }
      }
    },
    abort(cause) {
      // Clear up compressor sources on error/cancellation
    }
  };
}

For implementers, there is no Transformer protocol with begin(), rework(), flush() strategies and controller coordination handed right into a TransformStream class that has its personal hidden state machine and buffering mechanisms. Transforms are simply features or easy objects: far less complicated to implement and check.

When a bounded buffer fills up and a producer needs to put in writing extra, there are only some issues you are able to do:

1. Reject the write: refuse to simply accept extra information

2. Wait: block till area turns into out there

3. Discard previous information: evict what’s already buffered to make room

4. Discard new information: drop what’s incoming

That is it. Some other response is both a variation of those (like “resize the buffer,” which is de facto simply deferring the selection) or domain-specific logic that does not belong in a basic streaming primitive. Net streams at present all the time select Wait by default.

The brand new API makes you select one in all these 4 explicitly:

• strict (default): Rejects writes when the buffer is full and too many writes are pending. Catches “fire-and-forget” patterns the place producers ignore backpressure.

• block: Writes wait till buffer area is out there. Use if you belief the producer to await writes correctly.

• drop-oldest: Drops the oldest buffered information to make room. Helpful for stay feeds the place stale information loses worth.

• drop-newest: Discards incoming information when full. Helpful if you wish to course of what you’ve got with out being overwhelmed.

const { author, readable } = Stream.push({
  highWaterMark: 10,
  backpressure: 'strict' // or 'block', 'drop-oldest', 'drop-newest'
});

No extra hoping producers cooperate. The coverage you select determines what occurs when the buffer fills.

This is how every coverage behaves when a producer writes sooner than the patron reads:

// strict: Catches fire-and-forget writes that ignore backpressure
const strict = Stream.push({ highWaterMark: 2, backpressure: 'strict' });
strict.author.write(chunk1);  // okay (not awaited)
strict.author.write(chunk2);  // okay (fills slots buffer)
strict.author.write(chunk3);  // okay (queued in pending)
strict.author.write(chunk4);  // okay (pending buffer fills)
strict.author.write(chunk5);  // throws! too many pending writes

// block: Anticipate area (unbounded pending queue)
const blocking = Stream.push({ highWaterMark: 2, backpressure: 'block' });
await blocking.author.write(chunk1);  // okay
await blocking.author.write(chunk2);  // okay
await blocking.author.write(chunk3);  // waits till client reads
await blocking.author.write(chunk4);  // waits till client reads
await blocking.author.write(chunk5);  // waits till client reads

// drop-oldest: Discard previous information to make room
const dropOld = Stream.push({ highWaterMark: 2, backpressure: 'drop-oldest' });
await dropOld.author.write(chunk1);  // okay
await dropOld.author.write(chunk2);  // okay
await dropOld.author.write(chunk3);  // okay, chunk1 discarded

// drop-newest: Discard incoming information when full
const dropNew = Stream.push({ highWaterMark: 2, backpressure: 'drop-newest' });
await dropNew.author.write(chunk1);  // okay
await dropNew.author.write(chunk2);  // okay
await dropNew.author.write(chunk3);  // silently dropped

// Share with specific buffer administration
const shared = Stream.share(supply, {
  highWaterMark: 100,
  backpressure: 'strict'
});

const consumer1 = shared.pull();
const consumer2 = shared.pull(decompress);

As an alternative of tee() with its hidden unbounded buffer, you get specific multi-consumer primitives. Stream.share() is pull-based: shoppers pull from a shared supply, and also you configure the buffer limits and backpressure coverage upfront.

There’s additionally Stream.broadcast() for push-based multi-consumer situations. Each require you to consider what occurs when shoppers run at totally different speeds, as a result of that is an actual concern that should not be hidden.

Not all streaming workloads contain I/O. When your supply is in-memory and your transforms are pure features, async equipment provides overhead with out profit. You are paying for coordination of “waiting” that provides no profit.

The brand new API has full parallel sync variations: Stream.pullSync(), Stream.bytesSync(), Stream.textSync(), and so forth. In case your supply and transforms are all synchronous, you may course of your entire pipeline with no single promise.

// Async — when supply or transforms could also be asynchronous
const textAsync = await Stream.textual content(supply);

// Sync — when all parts are synchronous
const textSync = Stream.textSync(supply);

This is an entire synchronous pipeline – compression, transformation, and consumption with zero async overhead:

// Synchronous supply from in-memory information
const supply = Stream.fromSync([inputBuffer]);

// Synchronous transforms
const compressed = Stream.pullSync(supply, zlibCompressSync);
const encrypted = Stream.pullSync(compressed, aesEncryptSync);

// Synchronous consumption — no guarantees, no occasion loop journeys
const outcome = Stream.bytesSync(encrypted);

The complete pipeline executes in a single name stack. No guarantees are created, no microtask queue scheduling happens, and no GC strain from short-lived async equipment. For CPU-bound workloads like parsing, compression, or transformation of in-memory information, this may be considerably sooner than the equal Net streams code – which might drive async boundaries even when each part is synchronous.

Net streams has no synchronous path. Even when your supply has information prepared and your rework is a pure perform, you continue to pay for promise creation and microtask scheduling on each operation. Guarantees are implausible for instances during which ready is definitely vital, however they are not all the time vital. The brand new API permits you to keep in sync-land when that is what you want.

The async iterator primarily based strategy supplies a pure bridge between this various strategy and Net streams. When coming from a ReadableStream to this new strategy, merely passing the readable in as enter works as anticipated when the ReadableStream is ready as much as yield bytes:

const readable = getWebReadableStreamSomehow();
const enter = Stream.pull(readable, transform1, transform2);
for await (const chunks of enter) {
  // course of chunks
}

When adapting to a ReadableStream, a bit extra work is required because the various strategy yields batches of chunks, however the adaptation layer is as simply simple:

async perform* adapt(enter) {
  for await (const chunks of enter) {
    for (const chunk of chunks) {
      yield chunk;
    }
  }
}

const enter = Stream.pull(supply, transform1, transform2);
const readable = ReadableStream.from(adapt(enter));

• Unconsumed our bodies: Pull semantics imply nothing occurs till you iterate. No hidden useful resource retention. In the event you do not eat a stream, there is no background equipment holding connections open.

• The tee() reminiscence cliff: Stream.share() requires specific buffer configuration. You select the highWaterMark and backpressure coverage upfront: no extra silent unbounded development when shoppers run at totally different speeds.

• Rework backpressure gaps: Pull-through transforms execute on-demand. Knowledge would not cascade by way of intermediate buffers; it flows solely when the patron pulls. Cease iterating, cease processing.

• GC thrashing in SSR: Batched chunks (Uint8Array[]) amortize async overhead. Sync pipelines through Stream.pullSync() remove promise allocation completely for CPU-bound workloads.

The design decisions have efficiency implications. Listed here are benchmarks from the reference implementation of this potential various in comparison with Net streams (Node.js v24.x, Apple M1 Professional, averaged over 10 runs):

The chained rework result’s notably putting: pull-through semantics remove the intermediate buffering that plagues Net streams pipelines. As an alternative of every TransformStream eagerly filling its inner buffers, information flows on-demand from client to supply.

Now, to be honest, Node.js actually has not but put vital effort into totally optimizing the efficiency of its Net streams implementation. There’s doubtless vital room for enchancment in Node.js’ efficiency outcomes by way of a little bit of utilized effort to optimize the recent paths there. That mentioned, working these benchmarks in Deno and Bun additionally present a major efficiency enchancment with this various iterator primarily based strategy than in both of their Net streams implementations as properly.

Browser benchmarks (Chrome/Blink, averaged over 3 runs) present constant good points as properly:

These benchmarks measure throughput in managed situations; real-world efficiency relies on your particular use case. The distinction between Node.js and browser good points displays the distinct optimization paths every surroundings takes for Net streams.

It is value noting that these benchmarks evaluate a pure TypeScript/JavaScript implementation of the brand new API in opposition to the native (JavaScript/C++/Rust) implementations of Net streams in every runtime. The brand new API’s reference implementation has had no efficiency optimization work; the good points come completely from the design. A local implementation would doubtless present additional enchancment.

The good points illustrate how basic design decisions compound: batching amortizes async overhead, pull semantics remove intermediate buffering, and the liberty for implementations to make use of synchronous quick paths when information is out there instantly all contribute.

“We’ve done a lot to improve performance and consistency in Node streams, but there’s something uniquely powerful about starting from scratch. New streams’ approach embraces modern runtime realities without legacy baggage, and that opens the door to a simpler, performant and more coherent streams model.”
– Robert Nagy, Node.js TSC member and Node.js streams contributor

I am publishing this to begin a dialog. What did I get proper? What did I miss? Are there use instances that do not match this mannequin? What would a migration path for this strategy appear like? The purpose is to assemble suggestions from builders who’ve felt the ache of Net streams and have opinions about what a greater API ought to appear like.

A reference implementation for this various strategy is out there now and might be discovered at https://github.com/jasnell/new-streams.

• API Reference: See the API.md for full documentation

• Examples: The samples listing has working code for widespread patterns

I welcome points, discussions, and pull requests. In the event you’ve run into Net streams issues I have never coated, or in the event you see gaps on this strategy, let me know. However once more, the concept right here is to not say “Let’s all use this shiny new object!”; it’s to kick off a dialogue that appears past the present establishment of Net Streams and returns again to first ideas.

Net streams was an bold venture that introduced streaming to the online platform when nothing else existed. The individuals who designed it made affordable decisions given the constraints of 2014 – earlier than async iteration, earlier than years of manufacturing expertise revealed the sting instances.

However we have discovered loads since then. JavaScript has advanced. A streaming API designed at this time might be less complicated, extra aligned with the language, and extra specific concerning the issues that matter, like backpressure and multi-consumer conduct.

We deserve a greater stream API. So let’s discuss what that would appear like.