The Buffer class is a native global in Node.js designed for direct manipulation of binary data. Unlike browser-based JavaScript, which rarely deals with raw binary streams, Node.js operates as a server-side runtime where handling file systems, network packets, and I/O streams is standard. Buffer instances represent fixed-length memory allocations outside the V8 heap, offering high-performance access to raw data.

Creating Buffers

There are multipel ways to instantiate a buffer depending on the use case.

Allocating Space

Use Buffer.alloc to create a buffer of a specific size, initialized with zeros.

const emptyBuffer = Buffer.alloc(16);
console.log(emptyBuffer); // <Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>

Initializing from Data

Buffer.from allows creation from strings, arrays, or other buffers.

const numBuffer = Buffer.from([10, 20, 30]);
console.log(numBuffer); // <Buffer 0a 14 1e>

const textBuffer = Buffer.from("Node.js Guide");
console.log(textBuffer.toString()); // Node.js Guide

Practical Applications

Encoding and Decoding Text

Buffers are essential for converting text between different encodings like UTF-8, Hex, or Base64.

const payload = Buffer.from('Data Transfer', 'utf8');

console.log(payload.toString('hex')); 
// 44617461205472616e73666572

console.log(payload.toString('base64')); 
// RGF0YSBUcmFuc2Zlcg==

File System Interaction

The fs module utilizes buffers for reading and writing binary or text files efficiently.

const fs = require('fs');

const content = 'Sample text for file operations';
const fileBuffer = Buffer.from(content);

// Write to disk
fs.writeFile('output.log', fileBuffer, (error) => {
  if (error) console.error('Write failed', error);
  else console.log('Write successful');
});

// Read from disk
fs.readFile('output.log', (error, data) => {
  if (error) throw error;
  console.log('File content:', data.toString());
});

TCP Network Communication

When dealing with raw TCP sockets, data is transmitted as buffers.

const net = require('net');

// Server setup
const tcpServer = net.createServer((connection) => {
  connection.on('data', (segment) => {
    console.log('Incoming:', segment.toString());
    const reply = Buffer.from('ACK: Message Received');
    connection.write(reply);
  });
});

tcpServer.listen(4040, () => console.log('TCP Server active on 4040'));

// Client setup
const clientConn = net.connect({ port: 4040 }, () => {
  const message = Buffer.from('Ping');
  clientConn.write(message);
});

clientConn.on('data', (segment) => {
  console.log('Server says:', segment.toString());
  clientConn.end();
});

Streaming Data

Buffers are the fundamental unit of data in Streams, allowing large files to be processed piece by piece.

const fs = require('fs');

const reader = fs.createReadStream('source.txt');
const writer = fs.createWriteStream('destination.txt');

reader.on('data', (chunk) => {
  // chunk is already a Buffer
  writer.write(chunk);
});

reader.on('end', () => writer.end());

Manipulating Buffer Content

Writing and Slicing

You can write strings into a buffer and extract specific portions.

const storage = Buffer.alloc(128);
const written = storage.write('Buffer Manipulation', 'utf8');

console.log(`Bytes used: ${written}`);
console.log(storage.slice(0, written).toString());

Concatenation

Merge multiple buffers into a single instance using Buffer.concat.

const partA = Buffer.from('Hello ');
const partB = Buffer.from('There');
const fullMessage = Buffer.concat([partA, partB]);

console.log(fullMessage.toString()); // Hello There

Comparison

Compare buffers to determine sort order or equality.

const alpha = Buffer.from('XYZ');
const beta = Buffer.from('XYZ');
const gamma = Buffer.from('XYZZ');

console.log(alpha.compare(beta) === 0); // true
console.log(alpha.compare(gamma));      // -1 (alpha < gamma)