How to Easily Stream a Response Using Fastify and Node.JS

Node.JS is an excellent language to create APIs. It is lightweight, fast, and it's non-blocking nature allows it to handle many more requests at once than a synchronous language.

Streaming a HTTP response from a REST API is a great way to improve response times and decrease memory usage of the server.

Often, the flow during a request is: Parse incoming data (such as query parameters) » lookup information from a data source (such as querying a database) » load that data » send it back to the requestor.

This means your server needs to hold all that data from the database in memory until you send it back to the requestors. By utilizing streams, you eliminate this need by sending the data to the requestor as soon as your database gives it to you. Not only do you not use up memory, but it's faster!

Fastify is a low overhead, wicked fast, API framework for Node. Here is an example of how you can stream a MongoDB query back to the requestor.

// Modules you need
const MongoClient = require('mongodb').MongoClient
// This module will convert our objects from Mongo into strings
const stringify = require('streaming-json-stringify')
// You can use the stringify default stringifier, but this one is faster
const stringifier = require('fast-safe-stringify')

// Establish MongoDB connection
const mongo = MongoClient.connect('mongodb://my-mongo-url/my-db').collection('my-collection-name')

// This is the function being called by our Fastify; our route handler
function search (req, reply) {

  // This query returns all the items in the collection
  const queryStream = mongo.find({}).stream()

  // Fastify defaults streams to content-type of 'application/octet-stream', 
  // which will result in the downloading of the file. If you want to consume
  // it as JSON, we need to set the header manually
    .header('content-type', 'application/json')
    // Above, we send (pipe) our query stream to our stringify stream, which Fastify sends (pipes) 
    // to the response stream back to the user
Small dog web development studio

View All Our Posts

Web application developers in western Washington, creating user centric web solutions for businesses of all shapes and sizes.

Our projects include ecommerce, event organization, user driven web applications and more.

Small Dog Studios, LLC. Small Dog Studios News