MongoDB Oplog Fundamentals

The oplog (operation log) is a capped collection in MongoDB's local database that records all data modification operations. It serves as the foundation for replication in MongoDB replica sets, allowing secondary nodes to synchronize with the primary by replaying these operations.

Oplog Structure and Characteristics

Each oplog entry contains several key fields:

{
  ts: Timestamp(1634567800, 1),  // Operation timestamp
  h: NumberLong("1234567890"),   // Unique operation identifier
  op: "i",                       // Operation type (i=insert, u=update, d=delete)
  ns: "test.users",              // Namespace (database.collection)
  o: {                           // Operation document
    _id: ObjectId("..."),
    name: "John",
    age: 30
  }
}

Replica Set Synchronization Process

The replication workflow follows these steps:

1. Primary node executes write operations
2. Operations are recorded in the primary's oplog
3. Secondary nodes fetch and apply these operations
4. Applied operations are recorded in secondary oplogs

Oplog Size Management

Oplog size is critical for proper replication:

• Default size is 5% of available disk space (min 1GB, max 50GB)
• Can be configured using --oplogSize at startup
• Can be resized dynamically with replSetResizeOplog

Checking Oplog Status

Use these commands to monitor oplog:

// Get replication info
rs.printReplicationInfo()

// Check slave sync status
db.printSlaveReplicationInfo()

Practical Applications of Oplog

1. Data Replication

Secondary nodes use oplog to maintain data consistency with the primary node.

2. Change Monitoring

Applications can track database changes by reading the oplog stream.

3. Data Recovery

Oplog entries can be replayed to restore data to a previous state.

Querying Oplog Data

Recent Operations

db.getSiblingDB('local').oplog.rs
  .find()
  .sort({$natural: -1})
  .limit(5);

Time-based Filtering

const start = new Timestamp(1620000000, 0);
const end = new Timestamp(1620086400, 0);

db.getSiblingDB('local').oplog.rs
  .find({ts: {$gte: start, $lte: end}});

Namespace-specific Operations

db.getSiblingDB('local').oplog.rs
  .find({ns: 'inventory.products'});

Important Considerations

• Oplog operations are idempotent (safe to replay multiple times)
• Large batch operations may consume oplog space quickly
• Oplog doesn't record read operations
• Direct oplog manipulation is generally not recommended