Binary Data Handling with Buffers

In Node.js, the Buffer class provides a mechanism to handle raw binary data directly. Unlike standard arrays, a Buffer represents a fixed-size memory segment allocated outside the V8 heap. This is essential for operations involving TCP streams or file system interactions where data must be processed as byte sequences.

Buffers can be instantiated using several methods. Buffer.alloc(size) creates a zero-filled buffer, ensuring memory is clean, while Buffer.allocUnsafe(size) is faster but may contain old data remnants. You can also create buffers from existing strings or arrays.

// Allocating a clean memory block of 10 bytes
const secureBuffer = Buffer.alloc(10);

// Faster allocation with potential memory exposure
const fastBuffer = Buffer.allocUnsafe(10);

// Creating from a string
const dataBuffer = Buffer.from('Protocol Start');

Accessing and modifying buffer contents works similarly to arrays, but values are constrained to a single byte (0-255). Setting a value larger than 255 results in overflow, where only the least significant 8 bits are retained.

const byteSequence = Buffer.from('Data');
console.log(byteSequence[0].toString(2)); // Binary representation

byteSequence[0] = 88; // Modify byte
console.log(byteSequence.toString());

Computing Fundamentals and Execution Context

Understanding the runtime environment is crucial for backend development. The Central Processing Unit (CPU) executes instructions fetched from Random Access Memory (RAM). RAM provides high-speed read/write capabilities but is volatile, meaning data is lost upon power loss. In contrast, hard drives offer persistent storage but with significantly lower access speeds.

The Operating System (OS) acts as an intermediary, managing hardware resources and isolating running programs. A process is essentially a program in execution—an instance of a loaded application. Within each process, one or more threads exist as the smallest units of CPU scheduling, performing the actual computation tasks.

File System Interaction

The built-in fs module enables interaction with the hard drive, allowing programs to create, read, update, and delete files.

Writing and Appending Data

Basic file writing can be performed synchronously or asynchronously. While synchronous methods block the event loop, asynchronous methods use callbacks or Promises to handle I/O operations without halting execution.

const fs = require('fs');
const path = require('path');

// Asynchronous write
fs.writeFile(
    path.join(__dirname, 'log.txt'),
    'Initial log entry.',
    (err) => {
        if (err) throw err;
        console.log('File saved.');
    }
);

// Appending content using the flag option
fs.writeFile(
    path.join(__dirname, 'log.txt'),
    '\nAdditional entry.',
    { flag: 'a' },
    (err) => {
        if (err) throw err;
    }
);

Stream-Based Processing

For handling large files or continuous data inputs, streams are more efficient than reading entire files into memory. A WriteStream opens a file channel, allowing data to be written in chunks.

const output = fs.createWriteStream('large_data.bin');
output.write('Chunk 1 ');
output.write('Chunk 2 ');
output.end(); // Close the stream

Reading Files

Reading can be done in whole (readFile) or via streams (createReadStream). Streams are particularly useful for piping data from a source directly to a destination, such as copying a large video file.

const readStream = fs.createReadStream('source.mp4');
const writeStream = fs.createWriteStream('destination.mp4');

// Pipe manages flow control automatically
readStream.pipe(writeStream);

console.log('Stream copying initiated');

Directory and Resource Management

The fs module provides utilities for managing directories and inspecting file metadata. fs.mkdir creates directories, optionally recursively. fs.stat retrieves metadata such as file size, creation time, and type differentiation (file vs directory).

// Recursive directory creation
fs.mkdir('./data/backup', { recursive: true }, (err) => {
    if (err) console.error(err);
});

// Checking file status
fs.stat('./log.txt', (err, stats) => {
    if (stats) {
        console.log(`Is File: ${stats.isFile()}`);
        console.log(`Size: ${stats.size} bytes`);
    }
});

Path Manipulation

File paths often cause issues when code is moved across operating systems due to different separators (e.g., \ on Windows vs / on POSIX). The path module abstracts these differences, ensuring path consistency.

• path.resolve: Generates an absolute path by processing a sequence of paths from right to left.
• path.join: Concatenates path segments using the platform-specific separator.
• path.basename: Extracts the final segment of a path (the filename).
• path.extname: Returns the file extension.

const targetDir = '/user/docs';
const filename = 'notes.txt';

// Robust path construction
const fullPath = path.join(targetDir, filename);

// Extracting info
console.log(path.basename(fullPath)); // 'notes.txt'
console.log(path.extname(fullPath));  // '.txt'

// Safe absolute path resolution using __dirname
const safePath = path.resolve(__dirname, 'config.');