A definitive, production-grade beginner-to-expert technical guide and interactive sandbox for MongoDB 8 & Mongoose 8. This curriculum starts with zero-prerequisite document database concepts, BSON anatomy, and everyday CRUD operations, advances through WiredTiger storage engine internals and memory mechanics, dives deep into the Aggregation Framework and ESR index optimization, and culminates in multi-document ACID transactions, distributed sharding, and high-availability replica sets.
+-----------------------------------------------------------------------------------------------+
| THE MONGODB LEARNING JOURNEY |
+-------------------+-------------------+-----------------------+-------------------------------+
| STAGE 1 | STAGE 2 | STAGE 3 & 4 | STAGE 5, 6 & 7 |
| Absolute Beginner | Intermediate Core | Advanced Query & Index| Expert Scale & Staff Arch |
+-------------------+-------------------+-----------------------+-------------------------------+
| • What is NoSQL? | • WiredTiger Arch | • Aggregation Pipeline| • Multi-Doc ACID Transact. |
| • BSON Documents | • Cache & Evict. | • $match, $group, $lkp| • Sharded Clusters & Routers |
| • Collections | • Checkpoints/WAL | • ESR Indexing Rule | • Replica Sets & Oplog Sync |
| • insert, find | • Document Schema | • Compound & Multikey | • Mongoose 8 Middleware |
| • update, delete | • Relationships | • explain() Analysis | • 40 Staff Interview Q&A |
+-------------------+-------------------+-----------------------+-------------------------------+
- Stage 1: Absolute Beginner Foundations
- What is MongoDB? Relational vs Document Databases
- BSON vs JSON: The Binary Document Model
- Connecting via
mongosh& Essential CLI Commands - The
_idField &ObjectIdAnatomy - Basic CRUD Operations:
insertOne,find,updateOne,deleteOne - Query Operators & Filtering:
$gt,$in,$and,$or - Projections, Sorting & Pagination (
sort,skip,limit)
- Stage 2: Intermediate Core & WiredTiger Engine Architecture
- Stage 3: Aggregation Framework Masterclass
- Stage 4: Indexing Architecture & The ESR Performance Rule
- Stage 5: Data Modeling & Multi-Document ACID Transactions
- Stage 6: Replication, Sharding & Mongoose 8 ODM
- Stage 7: Staff & Principal MongoDB Interview Masterclass & Sandbox
MongoDB is a document-oriented NoSQL database engineered for agility, horizontal scale, and developer ergonomics.
In traditional relational databases (MySQL, PostgreSQL), data is divided across rigid tabular schemas requiring schema migrations (ALTER TABLE) whenever application models change. In MongoDB, data is stored in flexible, self-describing Documents grouped into Collections:
+---------------------------+-----------------------------------+
| Relational Concept (SQL) | MongoDB Concept (NoSQL) |
+---------------------------+-----------------------------------+
| Database (Schema) | Database |
| Table | Collection |
| Row (Tuple / Record) | Document (BSON) |
| Column (Attribute) | Field (Key-Value Pair) |
| Primary Key | `_id` Field (Default ObjectId) |
| Multi-Table JOIN | Embedded Subdocuments or `$lookup`|
| Foreign Key | Reference (`ObjectId`) |
+---------------------------+-----------------------------------+
While developers interact with MongoDB using familiar JSON syntax, MongoDB physically stores records as BSON (Binary JSON):
- Rich Extended Types: Standard JSON only supports strings, numbers, booleans, arrays, and null. BSON introduces exact floating-point numbers (
Decimal128), 64-bit integers (int64), native Date instances (ISODate), raw binary buffers, and regular expressions. - High-Performance Scanning: BSON prefixes every sub-element with its byte length, allowing the database engine to traverse or skip nested fields in memory without decoding every single character.
{
"_id": "65e8b4e72a88437f191b2c4e",
"name": "Alex Johnson",
"email": "alex@example.com",
"age": 31,
"isActive": true,
"tags": ["developer", "cloud"],
"address": {
"city": "Seattle",
"country": "USA"
},
"createdAt": "2026-03-01T10:00:00.000Z"
}Connect to MongoDB using the modern interactive shell mongosh:
# Connect to local default instance (port 27017)
mongosh
# Connect to a secure remote cluster with authentication
mongosh "mongodb://myuser:mypass@db.example.com:27017/store_db?authSource=admin"Inside the mongosh terminal, use these fundamental commands:
// List all physical databases on the server
show dbs
// Switch database context (auto-created on first insert if non-existent)
use store_db
// List all collections in the current database
show collections
// Query cluster status and replica set role (PRIMARY / SECONDARY)
rs.status()use store_db: In MongoDB, you do not need to pre-create databases or collections using explicit DDL commands. MongoDB lazily allocates the database and collection in WiredTiger storage upon inserting the first document.
Every document stored in MongoDB requires an immutable, unique primary key named _id. If omitted during document creation, the MongoDB driver or server automatically generates a 12-byte ObjectId:
+-------------------------------------------------------------------------+
| 12-BYTE OBJECTID ANATOMY |
+--------------------------+-----------------------+----------------------+
| 4-Byte Timestamp | 5-Byte Random Value | 3-Byte Counter |
| (Seconds since Unix epoch| (Process/Machine ID) | (Incrementing count) |
+--------------------------+-----------------------+----------------------+
- Timestamp Ordering: Because the first 4 bytes represent seconds since January 1, 1970,
ObjectIds are roughly chronological. You can extract the exact creation timestamp anytime via.getTimestamp():const id = ObjectId("65e8b4e72a88437f191b2c4e"); console.log(id.getTimestamp()); // Output: 2024-03-06T18:22:31.000Z
- Distributed Uniqueness: The 5-byte process identifier combined with the 3-byte monotonic counter guarantees that distributed client servers can generate unique IDs locally without round-trips to a centralized database sequence!
// 1. CREATE (Insert)
db.products.insertOne({
title: "Mechanical Keyboard",
sku: "TECH-001",
price: 129.99,
stock: 50,
tags: ["hardware", "peripherals"]
});
// 2. READ (Find)
// Retrieve all documents
db.products.find();
// Retrieve matching documents with filter
db.products.find({ sku: "TECH-001" });
// 3. UPDATE (Modify fields using atomic update operators)
// Always use $set to avoid overwriting the entire document!
db.products.updateOne(
{ sku: "TECH-001" },
{
$set: { price: 119.99 },
$inc: { stock: -1 } // Atomically decrement stock by 1
}
);
// 4. DELETE (Remove records)
db.products.deleteOne({ sku: "TECH-001" });db.products.insertOne({...}):- Validates BSON document constraints, appends the record into WiredTiger's in-memory cache, writes a record to the Write-Ahead Log (WAL journal) to guarantee Durability, and indexes the
_idfield in the default unique index.
- Validates BSON document constraints, appends the record into WiredTiger's in-memory cache, writes a record to the Write-Ahead Log (WAL journal) to guarantee Durability, and indexes the
db.products.updateOne({ filter }, { $set: ..., $inc: ... }):- Crucial Rule: In MongoDB, writing
updateOne({ sku: "TECH-001" }, { price: 119.99 })without$setwould replace the entire document with only{ price: 119.99 }, wiping out all other fields! Always use$setto update specific attributes in place. $inc: { stock: -1 }: Atomically decrements the value in place, avoiding race conditions in concurrent e-commerce checkout flows.
- Crucial Rule: In MongoDB, writing
// 1. Range Comparisons ($gte, $lte)
db.products.find({
price: { $gte: 50.00, $lte: 150.00 },
stock: { $gt: 0 }
});
// 2. Array Membership ($in)
db.products.find({
tags: { $in: ["hardware", "keyboards"] }
});
// 3. Compound Logical Disjunction ($or)
db.products.find({
$or: [
{ price: { $lt: 40.00 } },
{ stock: { $gt: 100 } }
]
});price: { $gte: 50, $lte: 150 }:- Queries documents where
price >= 50andprice <= 150. If an index exists onprice, WiredTiger executes a fast B-Tree range seek.
- Queries documents where
tags: { $in: [...] }:- When querying array fields, MongoDB automatically checks if any element inside the array matches the
$inlist, leveraging Multikey Indexes under the hood.
- When querying array fields, MongoDB automatically checks if any element inside the array matches the
// Query with selective projection, ordering, and pagination
db.products.find(
{ stock: { $gt: 0 } }, // Filter criteria
{ title: 1, price: 1, _id: 0 } // Projection: 1 = Include, 0 = Exclude
)
.sort({ price: -1 }) // Sort: -1 = Descending, 1 = Ascending
.skip(10) // Skip first 10 documents
.limit(5); // Take next 5 documents (Page 2, Size 5){ title: 1, price: 1, _id: 0 }(Projection):- Instructs MongoDB to omit unneeded fields from serialization, dramatically reducing network bandwidth. By default,
_idis always returned unless explicitly suppressed with_id: 0.
- Instructs MongoDB to omit unneeded fields from serialization, dramatically reducing network bandwidth. By default,
.sort({ price: -1 }):- Orders the returned stream. If a compound index on
(stock, price)exists, MongoDB uses index ordering, avoiding in-memory sort buffer memory limits (default 100MB).
- Orders the returned stream. If a compound index on
.skip(10).limit(5):- For high-offset pagination in production, avoid
.skip()because MongoDB must scan through all skipped documents. Use Range/Keyset Pagination (_id: { $lt: lastSeenId }) instead.
- For high-offset pagination in production, avoid
MongoDB is a document-oriented, distributed NoSQL database designed for horizontal scalability, high availability, and developer agility. Documents are stored in BSON (Binary JSON) format, allowing rich embedded hierarchical data structures, flexible schemas, and zero-impedance object-relational mapping.
+-----------------------------------------------------------------------+
| CLIENT LAYER |
| Node.js / Express / Mongoose Driver Connection Pool |
+-----------------------------------------------------------------------+
| (BSON Protocol over TLS)
v
+-----------------------------------------------------------------------+
| MONGODB CORE DATABASE ENGINE |
| +-----------------------+ +-------------------+ +---------------+ |
| | Query Parser & Opt | | Aggregation Pipe | | Auth & RBAC | |
| +-----------------------+ +-------------------+ +---------------+ |
+-----------------------------------------------------------------------+
|
v
+-----------------------------------------------------------------------+
| WIREDTIGER STORAGE ENGINE |
| +---------------------------+ +-----------------------------+ |
| | In-Memory Cache (RAM) | <---> | Eviction Server Threads | |
| | (Dirty & Clean Pages) | | (LRU / Hazard Pointers) | |
| +---------------------------+ +-----------------------------+ |
| | | |
| v (Checkpoints every 60s) v (WAL Journal) |
| +---------------------------+ +-----------------------------+ |
| | Data Files (B-Tree Blocks)| | Disk Journal (Write-Ahead) | |
| +---------------------------+ +-----------------------------+ |
+-----------------------------------------------------------------------+
While JSON is human-readable text, MongoDB uses BSON internally and on the wire:
- Binary Format: Lightweight, traversable, and fast to encode/decode into native language data types.
- Extended Types: Adds native support for
Date,ObjectId(12-byte globally unique timestamp-based identifiers),Binary Data(UUIDs, images),Decimal128(arbitrary-precision floating point for financial calculations),64-bit Integers (Long), andRegex. - Length Prefixing: Elements are prefixed with their byte length, allowing the query engine to skip sub-documents without parsing the entire payload.
WiredTiger has been the default storage engine since MongoDB 3.2. Understanding its memory management and persistence pipeline is essential for diagnosing latency spikes and memory starvation.
By default, MongoDB allocates the WiredTiger cache size using:
The remaining host memory is dedicated to OS page cache, query execution sorting memory (allowDiskUse), and connection overhead (approx. 1MB per open socket).
- Checkpoints: Every 60 seconds (or when 2GB of dirty data accumulates), WiredTiger flushes all modified pages from the in-memory cache to immutable disk blocks, creating a consistent snapshot on disk.
- Journal (WAL): Between 60-second checkpoints, durability is guaranteed by the Journal. Mutations are written sequentially to journal files on disk every 100ms (or immediately when a write specifies
j: true). If a crash occurs, MongoDB replays journal entries recorded since the last checkpoint to recover uncommitted transactions. - Eviction Triggers: Background eviction threads monitor dirty and clean memory thresholds:
- When cache utilization crosses 80%, background eviction begins.
- When dirty cache crosses 20%, background threads aggressively flush dirty pages.
- When cache exceeds 95%, application threads are stalled and forced to assist with eviction, causing severe query latency spikes.
- Node.js >= 20.x
- npm >= 10.x
- Docker & Docker Compose (optional for containerized MongoDB)
Clone the repository and install dependencies:
git clone https://github.com/manthanank/learn-mongodb.git
cd learn-mongodb
npm installCreate a local environment file .env:
PORT=3000
MONGODB_URI=mongodb://localhost:27017/learn-mongodb
# Or MongoDB Atlas connection string:
# MONGODB_URI=mongodb+srv://<username>:<password>@cluster0.mongodb.net/learn-mongodb?retryWrites=true&w=majorityNote on Offline Resilience: If no
MONGODB_URIis provided or if the cluster is unreachable,learn-mongodbautomatically falls back to an ultra-fast in-memory mock engine with pre-seeded enterprise documents.
# Start development server with live reload
npm run dev
# Run Vitest test suite
npm test
# Build production bundle
npm run build
# Start production server
npm start# Build production Docker image
docker build -t learn-mongodb:latest .
# Run containerized application
docker run -p 3000:3000 -e PORT=3000 learn-mongodb:latestMongoDB provides expressive querying capabilities with support for field comparison, logical evaluation, element projection, array inspection, and atomic update operators.
| Category | Operator | Description & Syntax |
|---|---|---|
| Comparison |
$eq, $ne
|
Matches values that are equal or not equal to a specified value. { age: { $eq: 30 } }
|
$gt, $gte
|
Greater than, greater than or equal. { salary: { $gte: 120000 } }
|
|
$lt, $lte
|
Less than, less than or equal. { age: { $lte: 45 } }
|
|
$in, $nin
|
Matches any / none of the values specified in an array. { department: { $in: ['Engineering', 'DevOps'] } }
|
|
| Logical |
$and, $or
|
Joins query clauses with a logical AND or OR. { $or: [{ status: 'A' }, { age: { $lt: 30 } }] }
|
$not, $nor
|
Inverts the effect of a query predicate or matches documents that fail all clauses. | |
| Element | $exists |
Matches documents that have the specified field. { phoneNumber: { $exists: true } }
|
$type |
Selects documents if a field is of the specified BSON type. { age: { $type: 'int' } }
|
|
| Evaluation | $regex |
Selects documents where values match a specified regular expression. { email: { $regex: /@enterprise\.io$/i } }
|
$expr |
Allows the use of aggregation expressions within the query language. { $expr: { $gt: ['$spent', '$budget'] } }
|
-
$all: Matches arrays that contain all elements specified in the query:db.users.find({ tags: { $all: ['mongodb', 'typescript'] } });
-
$elemMatch: Matches documents that contain an array field with at least one element that matches all the specified query criteria:db.orders.find({ items: { $elemMatch: { product: 'Server Rack', quantity: { $gte: 2 }, price: { $lt: 5000 } } } });
-
$size: Selects any document where the array field is a specified size:db.users.find({ tags: { $size: 3 } });
-
$set&$unset: Sets the value of a field or removes the specified field from a document:db.users.updateOne( { email: 'alex.morgan@enterprise.io' }, { $set: { department: 'Principal Architecture', updatedAt: new Date() } } );
-
$inc&$mul: Increments or multiplies the value of a field by a specified amount:db.users.updateMany( { department: 'Engineering' }, { $inc: { salary: 5000 } } );
-
Array Mutation Operators (
$push,$addToSet,$pull,$pop):// $addToSet prevents duplicate elements in an array db.users.updateOne( { _id: ObjectId('65e9b1f7d8e21a001a111111') }, { $addToSet: { tags: 'distributed-transactions' } } ); // $push with modifier options ($each, $slice, $sort) db.users.updateOne( { _id: ObjectId('65e9b1f7d8e21a001a111111') }, { $push: { auditLogs: { $each: [{ action: 'LOGIN', timestamp: new Date() }], $slice: -100 // Keep only the latest 100 entries } } } );
-
Positional Update Operators (
$[]and$[<identifier>]):// Filtered positional operator: update specific elements in nested arrays db.grades.updateMany( { studentId: 101 }, { $set: { 'scores.$[elem].curved': true } }, { arrayFilters: [{ 'elem.score': { $lt: 70 } }] } );
The MongoDB Aggregation Framework is a multi-stage data processing pipeline modeled on Unix pipes (|). Documents enter a multi-stage pipeline where each stage transforms the stream before passing results to the next.
+------------------+ +------------------+ +------------------+ +------------------+
| $match | --> | $group | --> | $lookup | --> | $sort |
| Filter documents | | Compute metrics | | Left outer join | | Order by metric |
+------------------+ +------------------+ +------------------+ +------------------+
| Stage | Purpose & Mechanics | Example Syntax |
|---|---|---|
$match |
Filters documents to pass only matching documents to the next stage. Place first to utilize indexes. | { $match: { status: 'ACTIVE', age: { $gte: 21 } } } |
$project |
Reshapes documents: includes, excludes, renames, or computes derived fields. | { $project: { fullName: { $concat: ['$firstName', ' ', '$lastName'] } } } |
$group |
Groups documents by an _id key and computes accumulators ($sum, $avg, $min, $max, $push). |
{ $group: { _id: '$dept', total: { $sum: 1 }, avgSalary: { $avg: '$salary' } } } |
$unwind |
Deconstructs an array field from the input documents to output a document for each element. | { $unwind: { path: '$tags', preserveNullAndEmptyArrays: false } } |
$lookup |
Performs an equality or correlated left outer join with another collection. | See correlated subquery syntax below. |
$facet |
Processes multiple aggregation pipelines within a single stage on the same input documents. | Enables multi-dimensional faceted search and pagination. |
$bucket |
Categorizes incoming documents into groups (buckets) based on specified boundaries. | { $bucket: { groupBy: '$age', boundaries: [20, 30, 40, 50] } } |
$addFields |
Outputs documents containing all existing fields plus newly computed fields. | { $addFields: { bonus: { $multiply: ['$salary', 0.15] } } } |
$merge |
Writes pipeline results directly to a target collection (on-demand materialized views). | { $merge: { into: 'monthly_summaries', whenMatched: 'replace' } } |
MongoDB 5.0+ allows expressive joins with pipeline subqueries, avoiding the limitation of simple equality matching:
db.orders.aggregate([
{ $match: { orderDate: { $gte: ISODate('2024-01-01') } } },
{
$lookup: {
from: 'inventory',
let: { orderItem: '$itemSku', orderQty: '$quantity' },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $eq: ['$sku', '$$orderItem'] },
{ $gte: ['$inStock', '$$orderQty'] }
]
}
}
},
{ $project: { _id: 0, warehouse: 1, inStock: 1 } }
],
as: 'stockAvailability'
}
}
]);Single-pass multi-faceted analytics using $facet:
db.products.aggregate([
{ $match: { category: 'Electronics', price: { $lte: 1000 } } },
{
$facet: {
// 1. Pagination & Data Slice
products: [
{ $sort: { rating: -1 } },
{ $skip: 20 },
{ $limit: 10 }
],
// 2. Total Count Metadata
metadata: [
{ $count: 'totalMatching' }
],
// 3. Price Histogram Distribution
priceDistribution: [
{
$bucket: {
groupBy: '$price',
boundaries: [0, 100, 250, 500, 1000],
default: 'Over 1000',
output: { count: { $sum: 1 } }
}
}
],
// 4. Top Brands
topBrands: [
{ $group: { _id: '$brand', count: { $sum: 1 } } },
{ $sort: { count: -1 } },
{ $limit: 5 }
]
}
}
]);-
Push Filters Early: Always place
$matchand$sortstages at the very beginning of the pipeline. MongoDB's query planner automatically collapses$matchinto the initial collection scan/index scan. - Limit Projection Overhead: If large documents with embedded payloads or images are present, project out unnecessary heavy fields early to minimize memory consumption between stages.
-
RAM Constraint (
allowDiskUse): Pipeline stages have a default 100MB RAM limit per stage (in MongoDB 6.0+, sorting stages overflow to disk if needed, but explicit{ allowDiskUse: true }guarantees predictable behavior on multi-gigabyte queries).
The MongoDB Aggregation Framework models data transformations as multi-stage pipelines where documents enter a multi-stage pipeline that transforms the documents into aggregated results.
flowchart LR
RawDocs["Raw Collection (1M Docs)"] --> M["$match (Filters with Index Scan)"]
M --> U["$unwind (Flattens nested arrays)"]
U --> G["$group (Calculates metrics by key)"]
G --> P["$project (Reshapes final JSON document)"]
P --> Out["$merge / $out (Stores into Materialized View)"]
// Filter active orders from 2026 and compute profit margin on the fly
db.orders.aggregate([
{
$match: {
status: "COMPLETED",
orderDate: { $gte: ISODate("2026-01-01") }
}
},
{
$project: {
orderId: 1,
customerEmail: "$customer.email",
totalRevenue: "$total",
profitMargin: {
$round: [
{ $multiply: [{ $divide: [{ $subtract: ["$total", "$cost"] }, "$total"] }, 100] },
2
]
}
}
}
]);// Flattens items array so each item becomes an independent top-level document
db.orders.aggregate([
{ $unwind: "$items" },
{
$group: {
_id: "$items.sku",
unitsSold: { $sum: "$items.quantity" },
totalGross: { $sum: { $multiply: ["$items.price", "$items.quantity"] } }
}
},
{ $sort: { totalGross: -1 } },
{ $limit: 5 }
]);// Joins reviews into products with complex filtering inside the sub-pipeline
db.products.aggregate([
{
$lookup: {
from: "reviews",
let: { productId: "$_id" },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $eq: ["$productId", "$$productId"] },
{ $gte: ["$rating", 4] } // Join only 4-star and 5-star reviews!
]
}
}
},
{ $sort: { createdAt: -1 } },
{ $limit: 3 }
],
as: "topPositiveReviews"
}
}
]);// Simultaneously computes price histogram, brand counts, and paginated items in 1 query!
db.products.aggregate([
{ $match: { category: "Electronics" } },
{
$facet: {
// Facet 1: Price Buckets
priceBuckets: [
{
$bucket: {
groupBy: "$price",
boundaries: [0, 50, 150, 500, 2000],
default: "Over $2000",
output: { count: { $sum: 1 } }
}
}
],
// Facet 2: Brand Distribution
brandCounts: [
{ $group: { _id: "$brand", count: { $sum: 1 } } },
{ $sort: { count: -1 } },
{ $limit: 5 }
],
// Facet 3: Paginated Results
paginatedResults: [
{ $sort: { rating: -1 } },
{ $skip: 0 },
{ $limit: 10 }
]
}
}
]);// Atomic field modifications and array manipulations
db.users.updateOne(
{ _id: ObjectId("65e0123456789abcdef01234") },
{
// Scalar modifiers
$set: { "profile.bio": "Full-Stack Software Architect" },
$inc: { "stats.loginCount": 1, "wallet.balance": -15.50 },
$min: { "stats.fastestResponseMs": 45 },
$max: { "stats.peakConcurrentUsers": 1200 },
$currentDate: { updatedAt: true },
// Array modifiers
$addToSet: { roles: "ADMIN" }, // Adds only if not already present!
$push: {
recentLogins: {
$each: [new Date()],
$slice: -10 // Keeps only the 10 most recent entries!
}
}
}
);Indexes in MongoDB are stored as balanced B-Trees (B-Trees in WiredTiger maintain ordered pointer keys pointing directly to disk record IDs). Without indexes, every query results in a full collection scan (COLLSCAN), loading every block from disk into RAM.
+-----------------------------+
| Root Page: ["M" - "S"] |
+-----------------------------+
/ | \
+---------------+ +----------------+ +---------------+
| Internal Node | | Internal Node | | Internal Node |
| ["A" - "L"] | | ["M" - "R"] | | ["S" - "Z"] |
+---------------+ +----------------+ +---------------+
/ \ / \ / \
+----+ +----+ +----+ +----+ +----+ +----+
|Leaf| |Leaf| |Leaf| |Leaf| |Leaf| |Leaf|
+----+ +----+ +----+ +----+ +----+ +----+
| | | | | |
[Doc1] [Doc2] [Doc3] [Doc4] [Doc5] [Doc6]
When creating compound indexes for queries that filter on equality, sort results, and filter across ranges, the order of fields in the compound index must follow the ESR Rule:
-
Equality (
E): Place fields queried with exact matches (field: valueor$eq) first. This narrows down the search space to a contiguous sub-tree. -
Sort (
S): Place fields used in thesort()criteria next. Because the index maintains keys in sorted order, the storage engine can return documents directly in the requested order without triggering an expensive, in-memory blockingSORTstage. -
Range (
R): Place fields queried with inequalities ($gt,$gte,$lt,$lte,$in,$regex) last. Once a range scan begins on a B-tree, subsequent index keys cannot be leveraged for ordering!
Consider the query:
db.users.find({ department: 'Engineering', age: { $gte: 25 } })
.sort({ salary: -1 });| Index Specification | Index Evaluation | Execution Plan Mechanics |
|---|---|---|
{ age: 1, salary: -1, department: 1 } |
❌ Anti-Pattern (R-S-E) | Scans wide range of age. Because age is a range, the B-tree order of salary is broken. Forces a blocking in-memory SORT stage and examines unnecessary documents. |
{ department: 1, age: 1, salary: -1 } |
Filters department rapidly, but because age (range) precedes salary (sort), MongoDB still cannot use the index for ordering. Results in SORT stage. |
|
{ department: 1, salary: -1, age: 1 } |
✅ Optimal (E-S-R) | Equality filters to Engineering. The engine traverses directly along pre-sorted salary keys in descending order, checking age >= 25 as it streams. Zero in-memory sort required! |
Partial indexes index only documents that meet a specified filter expression, drastically reducing index memory footprint and write overhead:
// Only index active users, skipping millions of archived/deleted records
db.users.createIndex(
{ email: 1 },
{ partialFilterExpression: { isActive: true } }
);Automatically purge temporary documents (sessions, OTP codes, audit caches) after a specified duration:
// Document will be automatically removed 3600 seconds (1 hour) after createdAt
db.sessions.createIndex(
{ createdAt: 1 },
{ expireAfterSeconds: 3600 }
);When an index field holds an array, MongoDB creates an index entry for every element in the array:
db.users.createIndex({ tags: 1 });
// Restriction: A compound multikey index cannot index more than one array field!// Full-text search index across multiple text fields with weights
db.articles.createIndex(
{ title: 'text', content: 'text' },
{ weights: { title: 10, content: 1 } }
);
// 2dsphere index for geospatial distance calculations
db.places.createIndex({ location: '2dsphere' });To diagnose query performance, inspect the execution plan:
db.users.find({ department: 'Engineering', age: { $gte: 25 } })
.sort({ salary: -1 })
.explain('executionStats');Key Metrics to Check:
stage: You want to seeIXSCAN(Index Scan) followed byFETCH, or ideallyPROJECTION_COVERED. Red flags:COLLSCAN(full table scan) andSORT(in-memory blocking sort).totalDocsExaminedvsnReturned: In an optimal query, the ratio is 1.0 (every document fetched from disk was returned to the client). A ratio > 10.0 indicates missing or poorly ordered index keys.totalKeysExamined: Number of B-tree entries traversed. IftotalKeysExaminedis much larger thannReturned, the index range is too wide.
In document databases, schema design is governed by application access patterns rather than theoretical third normal form (3NF). Data that is queried together should be stored together.
| Criterion | Embed Subdocuments | Reference Documents (_id) |
|---|---|---|
| Relationship Cardinality | 1-to-1, 1-to-Few (e.g., User addresses, items in an invoice) | 1-to-Many (10,000+ items), 1-to-Squillions (logs, sensor telemetry) |
| Access Pattern | Data is read and updated atomically in a single operation | Referenced data is queried independently or in separate modules |
| Document Size | Must remain well under the 16MB BSON limit | Prevents unbounded document growth |
| Consistency Requirements | Requires atomic single-document updates without transactions | References can use multi-document ACID transactions if needed |
+-----------------------------------------------------------------------------------------+
| ENTERPRISE SCHEMA PATTERNS |
| |
| [SUBSET PATTERN] [BUCKET PATTERN] [OUTLIER PATTERN] |
| Embed top 10 reviews only; Group IoT / time-series Flag popular entities to |
| offload history to separate metrics into 1-hour chunks; prevent unbounded array |
| overflow collection. reduces index RAM footprint. growth beyond 16MB. |
+-----------------------------------------------------------------------------------------+
Problem: An e-commerce product has 20,000 reviews. Loading the product document requires reading megabytes of reviews that users rarely scroll through.
Solution: Embed only the top 10 most helpful or recent reviews in the products document. Store the full review history in a dedicated reviews collection.
Problem: Storing one document per sensor measurement creates millions of documents, exploding index sizes and metadata overhead. Solution: Group measurements into 1-hour or 1-day "buckets" containing a fixed-size array of data points with pre-aggregated rolling metrics:
{
sensorId: 'TEMP_US_WEST_01',
date: ISODate('2024-03-01T00:00:00Z'),
count: 60,
minTemp: 18.2,
maxTemp: 24.5,
readings: [
{ minute: 0, temp: 18.5 },
{ minute: 1, temp: 18.6 }
]
}Problem: Most authors have 1 to 5 books, but Stephen King has published 80+. Sizing arrays for edge cases harms performance for 99.9% of typical users.
Solution: Set a ceiling on embedded arrays (e.g., 50 items). When an entity exceeds this threshold, set hasOverflow: true and direct subsequent writes to an overflow collection.
Problem: Altering schemas in large relational databases requires locking tables with expensive migrations.
Solution: Add a schemaVersion: 2 integer field to every document. Application code can support multiple versions concurrently without requiring instant offline database migrations.
Since version 4.0 (replica sets) and 4.2 (sharded clusters), MongoDB supports full Multi-Document ACID Transactions across multiple collections, databases, and shards.
import mongoose from 'mongoose';
async function transferFunds(fromUserId: string, toUserId: string, amount: number): Promise<void> {
const session = await mongoose.startSession();
try {
session.startTransaction({
readConcern: { level: 'snapshot' },
writeConcern: { w: 'majority', j: true },
maxCommitTimeMS: 5000
});
// Step 1: Deduct from sender
const sender = await UserModel.findOneAndUpdate(
{ _id: fromUserId, balance: { $gte: amount } },
{ $inc: { balance: -amount } },
{ session, new: true }
);
if (!sender) {
throw new Error('Insufficient funds or sender not found');
}
// Step 2: Credit receiver
const receiver = await UserModel.findByIdAndUpdate(
toUserId,
{ $inc: { balance: amount } },
{ session, new: true }
);
if (!receiver) {
throw new Error('Receiver account not found');
}
// Step 3: Commit transaction across replica set
await session.commitTransaction();
} catch (error) {
// Step 4: Abort on failure
await session.abortTransaction();
throw error;
} finally {
session.endSession();
}
}Specifies the level of acknowledgement requested from MongoDB for write operations:
w: 1: Write is acknowledged as soon as the Primary node writes to its memory cache. Risk: If the primary crashes before replicating to secondaries, the write is lost.w: 'majority': Write is acknowledged only after a majority of voting replica set members have received and committed the write to their memory. Guarantees safety against rollbacks.j: true(Journal): Write is acknowledged only after being written to the on-disk Write-Ahead Log (WAL) journal, guaranteeing crash durability.wtimeout: Prevents writes from blocking indefinitely if replica nodes are unreachable.
"local": Returns the node's most recent data without verifying if it was committed to a majority."majority": Returns data that has been written to and acknowledged by a majority of nodes. Immune to failover rollbacks."linearizable": Returns data that reflects all successfully acknowledged majority writes prior to the read. Primary communicates with peers before answering, eliminating stale reads."snapshot": Used in transactions. Guarantees a consistent snapshot view across all reads.
primary: All reads go to the Primary (strict consistency).primaryPreferred: Reads from Primary if available, otherwise Secondaries.secondary: All reads go to Secondaries (offloading read traffic, eventual consistency).secondaryPreferred: Reads from Secondaries, falling back to Primary.nearest: Reads from the member with the lowest network latency.
A Replica Set in MongoDB is a cluster of mongod instances that maintain the exact same dataset, providing high availability, redundancy, and automated failover.
+--------------------------------+
| PRIMARY NODE |
| Accepts all Writes & Reads |
| Writes to Oplog (local.oplog) |
+--------------------------------+
/ \
Replication / \ Replication
(Async Stream)/ \ (Async Stream)
v v
+--------------------+ +--------------------+
| SECONDARY NODE | | SECONDARY NODE |
| Replays OpLog | <----------> | Replays OpLog |
| Can serve Reads | Heartbeats | Can serve Reads |
+--------------------+ (Every 2s) +--------------------+
- Primary Node: The only node that accepts write operations. Records all mutations sequentially into its capped Oplog (
local.oplog.rs). - Secondary Nodes: Continuously pull and apply oplog entries from the primary asynchronously.
- Failover & Elections: Nodes exchange heartbeats every 2 seconds. If a primary fails to respond for 10 seconds, secondaries initiate a Raft-inspired election. The candidate with the most up-to-date oplog is elected primary.
- Arbiter Nodes: Voting-only members that hold no data. Discouraged in modern architectures; prefer dedicated odd-numbered data nodes (3, 5, or 7 nodes).
When data exceeds disk capacity or write throughput saturates a single primary, MongoDB splits collections across multiple independent replica sets (Shards).
+-----------------------------------------------------------------------+
| CLIENT / DRIVER |
+-----------------------------------------------------------------------+
|
v
+-----------------------------------------------------------------------+
| MONGOS (ROUTING / QUERY TIER) |
| Evaluates query filters against cached chunk routing tables. |
+-----------------------------------------------------------------------+
| |
v v
+-------------------------------+ +---------------------------------+
| CONFIG SERVER REPLICA SET | | SHARD REPLICA SETS |
| Holds cluster metadata & | | Shard 1: Data Chunks [A - M] |
| chunk ranges mapping table. | | Shard 2: Data Chunks [N - Z] |
+-------------------------------+ +---------------------------------+
The shard key determines how documents are distributed across shards:
- High Cardinality: Must have many distinct values. Sharding on
country(low cardinality) creates jumbo chunks that cannot be split across shards. - Even Frequency Distribution: Avoid keys where 90% of documents share one value.
- Avoid Monotonically Increasing Keys: Sharding by auto-incrementing IDs or timestamps creates a hotspot, routing 100% of inserts to a single shard!
- Hashed Sharding vs Range Sharding:
- Hashed Sharding: Distributes inserts evenly across all shards, eliminating write hotspots. (Drawback: Range queries become scatter-gather).
- Range Sharding: Clusters contiguous values on the same shard, optimizing range queries.
Mongoose provides schema-based modeling for MongoDB with type inference, lifecycle hooks, and validation.
import mongoose, { Schema, InferSchemaType, Model } from 'mongoose';
// 1. Define Schema
const customerSchema = new Schema(
{
companyName: { type: String, required: true, trim: true },
tier: { type: String, enum: ['Standard', 'Enterprise'], default: 'Standard' },
annualSpend: { type: Number, default: 0, min: 0 },
billingEmail: {
type: String,
required: true,
lowercase: true,
trim: true
}
},
{
timestamps: true,
toJSON: { virtuals: true },
toObject: { virtuals: true }
}
);
// 2. Infer TypeScript Type from Schema definition
export type ICustomer = InferSchemaType<typeof customerSchema>;
// 3. Document Lifecycle Middleware (Pre/Post Hooks)
customerSchema.pre('save', function (next) {
if (this.annualSpend >= 100000 && this.tier !== 'Enterprise') {
this.tier = 'Enterprise';
}
next();
});
// 4. Virtual Property
customerSchema.virtual('isHighValue').get(function (this: ICustomer) {
return this.annualSpend > 50000;
});
export const Customer: Model<ICustomer> = mongoose.model<ICustomer>('Customer', customerSchema);- SCRAM (Salted Challenge Response): Default authentication mechanism (
SCRAM-SHA-256) preventing plaintext password transmission. - x.509 Certificates: Enterprise authentication using TLS client certificates for mutual authentication.
- Role-Based Access Control (RBAC): Follow the principle of least privilege:
db.createUser({ user: 'appServiceUser', pwd: 'SuperSecurePasswordHere!', roles: [ { role: 'readWrite', db: 'production_app' }, { role: 'read', db: 'inventory_cache' } ] });
- In-Flight Encryption: Mandatory TLS 1.3 encryption across all client-to-cluster and inter-node replica communications.
- Client-Side Field Level Encryption (CSFLE): Encrypts sensitive fields (SSNs, credit cards) locally on the client driver before sending over the network, ensuring the database engine never sees plaintext data.
- Queryable Encryption (QE): MongoDB 6.0+ allows running equality and range queries on encrypted data fields without decrypting them on the server!
- Logical Backups (
mongodump/mongorestore): Exports BSON data files. Best for development and smaller datasets.mongodump --uri="mongodb://localhost:27017/learn-mongodb" --out=/backup/$(date +%F) mongorestore --uri="mongodb://localhost:27017/learn-mongodb" /backup/2024-03-01/learn-mongodb
- Physical Continuous Snapshots: In enterprise and Atlas deployments, disk volume snapshots with point-in-time recovery (PITR) replay the oplog to restore the exact state to any second.
Answer: WiredTiger maintains an in-memory cache sized by default to:
$$\text{WiredTiger Cache} = \max(500\text{MB}, 0.5 \times (\text{Total RAM} - 1\text{GB}))$$ Pages are held in memory in uncompressed format. When cache usage exceeds the dirty target ($5%\text{--}20%$ ) or maximum threshold ($80%$ ), background eviction server threads compress dirty pages and write them to disk. If cache fills to$95%$ , client application threads are forced to perform evictions, causing severe query latency spikes.
Answer:
- Replica Set Transactions (MongoDB 4.0+): Uses WiredTiger's storage engine snapshots. The primary holds locks and buffers write operations in memory until
commitTransaction(), where the changes are atomically applied and written to theoplog.- Sharded Cluster Transactions (MongoDB 4.2+): Uses a Two-Phase Commit (2PC) protocol coordinated by the
mongosrouter across participating shard primaries.
Answer:
w: 1: Returns acknowledgment as soon as the Primary has written the document to its local memory/journal. Fast, but vulnerable to data loss if the primary crashes before replicating to secondaries.w: "majority": Blocks acknowledgment until a strict mathematical majority ($>50%$ ) of voting replica set members have committed the write to their memory/journal, guaranteeing that the committed write cannot be rolled back upon primary failover.
Answer: If a Primary accepts a write under
w: 1and abruptly crashes before secondaries replicate it, a new Primary is elected. When the old Primary rejoins as a secondary, it realizes its local data diverged from the new Primary. It rolls back the un-replicated writes and writes the discarded BSON documents to disk in therollback/directory (<collection>.<timestamp>.bson).
Answer: When designing compound indexes for queries containing equality, sorting, and range conditions:
- Equality (
E): Place exact-match fields first ({ status: "ACTIVE" }).- Sort (
S): Place sort fields second ({ createdAt: -1 }). This allows the database to return rows in pre-sorted B-Tree order without performing an in-memory sort!- Range (
R): Place range fields last ({ age: { $gte: 21 } }).
Answer: If a query's
sort()condition cannot use an index, MongoDB must load all matching documents into RAM and sort them. MongoDB enforces a strict$100\text{MB}$ memory limit on in-memory sorts (or$32\text{MB}$ in older versions). If exceeded withoutallowDiskUse: true, the query crashes with an error.
Answer: A query is covered when:
- All fields in the query predicate (
find()) are part of an index.- All fields returned in the projection (
project) are part of the same index.- The
_idfield is explicitly suppressed ({ _id: 0 }). MongoDB resolves the query directly from the WiredTiger index B-Tree without fetching documents from disk, achieving maximum throughput.
Answer:
primary(Default): All reads go strictly to Primary. Guaranteed read-your-own-writes consistency.primaryPreferred: Reads from Primary; falls back to Secondaries during primary election failover.secondary: All reads routed to Secondaries. Good for offloading heavy analytics, but subject to replication lag (stale reads).secondaryPreferred: Reads from Secondaries; falls back to Primary if no secondaries are available.nearest: Measures network ping latency and routes to the closest node regardless of role.
Answer:
readConcern: "majority": Returns data that has been acknowledged by a majority of nodes (immune to rollbacks). Reads from an in-memory snapshot.readConcern: "linearizable": The primary checks in real time with a majority of nodes before returning the response to ensure it hasn't been deposed in a network partition split-brain. Guarantees true real-time serializability at the expense of network latency.
Answer: The
oplog.rs(Operations Log) is a capped collection in thelocaldatabase on every replica set member. When writes occur on the Primary, it records idempotent transformation instructions. Secondary nodes continuously tail the Primary's oplog using tailable cursors and apply changes locally in parallel.
Answer: A sharded cluster consists of:
mongosQuery Routers: Stateless gateways that parse client queries and route them to target shards.- Config Database (Replica Set): Stores cluster metadata and shard key chunk ranges.
- Shards (Replica Sets): Independent database clusters storing subsets of data partitioned by Shard Key.
Answer:
- Ranged Sharding: Partitions by contiguous values of the shard key. Excellent for range queries (
$gte,$lte), but vulnerable to hotspotting if the shard key is monotonically increasing (e.g._idor timestamp, where all new writes hit a single shard).- Hashed Sharding: Hashes the shard key with MD5. Distributes writes uniformly across all shards, eliminating write hotspots, but forces all range queries to scatter-gather across the entire cluster.
Answer: When a client executes a query that does NOT include the Shard Key in its filter, the
mongosrouter cannot deduce which shard holds the data. It is forced to broadcast ("scatter") the query to every single shard in the cluster, collect the results ("gather"), and merge them in memory, creating severe network and CPU bottlenecks.
Answer: Change Streams allow applications to subscribe to real-time data changes on a collection, database, or entire deployment. Built on top of the oplog with resume tokens, Change Streams automatically notify backend services when documents are inserted, updated, or deleted.
Answer: A fixed-size circular collection that preserves insertion order. When the allocated disk quota is reached, MongoDB automatically overwrites the oldest documents without index fragmentation or file system allocation overhead. Used for high-throughput audit logs and caching.
Answer: A single-field index on a date field with the
expireAfterSecondsoption. A background thread running every 60 seconds scans the index and automatically purges expired documents from the collection. Ideal for user sessions, auth tokens, and temporary verification codes.
Answer:
$push: Appends the value to the array unconditionally, allowing duplicates.$addToSet: Treats the array as a mathematical set; appends the value only if it does not already exist in the array.
Answer: Single BSON documents are capped at
$16\text{MB}$ to prevent excessive RAM consumption and network serialization latency. To store files or datasets exceeding 16MB:
- Use GridFS: Splits files into
$255\text{KB}$ binary chunks stored across two collections (fs.filesandfs.chunks).- Use Object Storage (AWS S3 / GCS) and store the S3 URL in the MongoDB document.
Answer: Storing 1 document per second for 1,000 sensors produces 86.4 million documents daily, ballooning index and BSON overhead. The Bucket Pattern aggregates readings for a time window (e.g., 1 hour) into an array inside a single document:
{ sensorId: 1, hour: ISODate(...), readings: [{ t: 1, v: 24.5 }, ...] }, reducing document count by$3600\times$ and improving query cache locality.
Answer: Used when most documents have small arrays (e.g. a book having 20 reviews), but a rare "outlier" has 100,000 reviews (e.g. Harry Potter). Storing all reviews embedded blows past the 16MB limit.
- Remediation: Embed reviews up to 100 in the main document and set an
hasOverflow: trueflag. Outlier reviews beyond 100 are stored in a separatereviewscollection.
Answer:
- Sparse Index: Indexes only documents that contain the indexed field, but does not support filtering expressions.
- Partial Index (Recommended): Indexes only documents matching a specified filter expression (
partialFilterExpression: { rating: { $gt: 4 } }). Consumes less disk space and memory while speeding up filtered queries.
Answer: Collation allows language-specific rules for string comparison and sorting (lettercase, accent marks). Specifying
{ locale: "en", strength: 2 }performs case-insensitive and accent-insensitive index comparisons.
Answer:
$out: Replaces the entire target collection with the aggregation output. Destructive; drops existing indexes and data.$merge(MongoDB 4.2+): Flexible output stage. Can output to existing collections, across databases, or on sharded collections. SupportswhenMatched: "replace" | "keepExisting" | "merge", enabling incremental materialized views!
Answer: Atlas Search embeds an Apache Lucene full-text search engine directly alongside the MongoDB WiredTiger database, enabling fuzzy search, autocompletion, facet filtering, and relevance scoring without syncing data to external Elasticsearch clusters.
Answer: Deploy a 5-node replica set distributed across 3 Availability Zones or cloud regions:
- Region 1 (Primary Datacenter): 2 voting data nodes (Priority 2 and 1).
- Region 2 (Secondary Datacenter): 2 voting data nodes (Priority 0.5).
- Region 3 (Quorum Tiebreaker): 1 lightweight Arbiter or voting node. If Region 1 experiences a total datacenter outage, Region 2 and Region 3 form a majority (3 out of 5 votes), automatically elect a new Primary in Region 2, and resume serving traffic in under 5 seconds.
Q1: What is the difference between BSON and JSON, and why does MongoDB use BSON?
Answer: JSON is a human-readable, text-based serialization format that supports only basic primitives (string, number, boolean, array, object, null). BSON (Binary JSON) is a binary-encoded serialization format that:
- Adds native support for rich enterprise types like
Date,ObjectId,Decimal128(arbitrary-precision financial decimals),Binary(UUIDs, raw bytes), and64-bit Integers. - Stores length prefixes for strings and nested documents, enabling the query engine to rapidly skip irrelevant subtrees in memory without parsing every character.
- Is optimized for fast traversal and minimal encoding/decoding overhead in client drivers.
Q2: How does the WiredTiger cache work, and what happens when dirty cache exceeds thresholds?
Answer: By default, WiredTiger reserves 50% of (RAM - 1GB) for its working cache. Memory holds clean pages (read from disk) and dirty pages (modified in memory).
- Checkpoints occur every 60 seconds (or when 2GB dirty data accumulates) to write consistent snapshots to disk.
- When cache usage exceeds 80%, background eviction threads wake up to write dirty pages or discard clean pages.
- If cache reaches 95% or dirty data exceeds 20%, client application threads are forced to assist in eviction, causing catastrophic query latency spikes.
Q3: Explain the ESR (Equality, Sort, Range) rule in compound index design.
Answer: When building a compound index for queries with filtering, sorting, and range conditions:
-
Equality (
E): Exact match fields (field: value) must come first to partition the B-tree into contiguous buckets. -
Sort (
S): Sorting keys must come second so that the storage engine traverses documents in sorted order directly from the index, eliminating an in-memory blockingSORTstage. -
Range (
R): Range filters ($gt,$lt,$in) must come last. Traversing a range breaks the ordering of subsequent index keys in the B-tree.
Q4: What is the 16MB document size limit in MongoDB, and what architectural patterns solve it?
Answer: MongoDB limits individual BSON documents to 16MB to prevent unbounded memory consumption and excessive network serialization delays. Architectural patterns to bypass this include:
-
GridFS: Splits files or binary payloads into chunks (default 255KB) across two collections (
fs.filesandfs.chunks). -
The Subset Pattern: Embeds only the latest
$N$ entries (e.g. 10 reviews) in the main document, storing the rest in a child collection. - The Bucket Pattern: Groups time-series or sensor readings into discrete documents spanning 1 hour or 1 day.
Q5: What is a Multikey index, and what are its critical limitations?
Answer: A multikey index is created when an indexed field contains an array. MongoDB generates an index entry for every element in that array.
- Limitation: A compound multikey index cannot contain more than one array field (e.g., you cannot index
{ tags: 1, categories: 1 }if bothtagsandcategoriesare arrays in the same document), because it would produce an exponential Cartesian product of index entries.
Q6: How does MongoDB achieve high availability in a Replica Set during Primary node failure?
Answer: Replica set members exchange heartbeats every 2 seconds. If secondaries miss heartbeats from the primary for 10 seconds, an election is triggered:
- A secondary node with election priority > 0 and the highest oplog optime nominates itself.
- It requests votes from all reachable members.
- If it secures a strict majority of votes (e.g.
$\lfloor N/2 \rfloor + 1$ ), it transitions to Primary. - If network partitioning prevents a majority, the cluster becomes read-only until quorum is restored.
Q7: What is the difference between Write Concern `w: 1` and `w: "majority"`?
Answer:
w: 1: Write is acknowledged as soon as the Primary commits it to its local in-memory cache. If the Primary crashes before replicating to secondaries, the committed write will be lost on failover.w: "majority": Write is acknowledged only after a strict majority of voting replica set members have received and committed the write to their local storage. This guarantees zero write loss and immunity to rollbacks.
Q8: What is the Journal in MongoDB, and what is the role of `j: true`?
Answer: The Journal is a sequential Write-Ahead Log (WAL) on disk. Checkpoints flush cache to disk only every 60s; in the interim, mutations are appended to the journal every 100ms. Specifying j: true forces the engine to flush the journal buffer to physical disk before acknowledging the write, guaranteeing crash durability.
Q9: How do Multi-Document ACID Transactions work in MongoDB?
Answer: Available in replica sets (4.0+) and sharded clusters (4.2+), multi-document transactions use client sessions (startSession()).
- All operations inside the transaction use Snapshot Isolation (
readConcern: 'snapshot'). - Changes are held in a transaction buffer until committed.
- When
commitTransaction()is invoked, a two-phase commit is orchestrated across all participating shards and replica set nodes with write concernw: "majority".
Q10: What is the difference between `$lookup` with simple equality vs `$lookup` with pipeline?
Answer:
- Simple
$lookup(localField/foreignField): Performs a basic left outer join on single matching field equalities. - Pipeline
$lookup(usingletandpipeline): Allows correlated subqueries with complex join conditions ($expr), filtering, projections, sorting, and nested aggregations on the foreign collection before returning matched documents.
Q11: Explain the Oplog (`local.oplog.rs`) and how secondaries sync data.
Answer: The Oplog is a capped collection containing idempotent records of all data mutations on the primary node. Secondaries tail the primary's oplog using tailable cursors and replay operations asynchronously. Because all oplog operations are idempotent, replaying them multiple times yields the exact same state.
Q12: What is the difference between Range Sharding and Hashed Sharding?
Answer:
-
Range Sharding: Divides data into contiguous ranges based on the shard key values. Documents with similar keys reside on the same shard, optimizing range queries (
$gte,$lte), but risking write hotspots if keys are monotonically increasing. - Hashed Sharding: Uses MD5 hashes of the shard key to scatter documents uniformly across all shards, eliminating write hotspots at the cost of turning range scans into scatter-gather queries.
Q13: Why is a monotonically increasing field (e.g. `ObjectId` or timestamp) bad as a range shard key?
Answer: Every new document has a value greater than all existing documents. Under range sharding, the max-range chunk will always reside on one single shard, routing 100% of all insert traffic to that one machine while all other shards sit idle.
Q14: What is a Covered Query in MongoDB?
Answer: A query where:
- All fields in the query predicate are part of an index.
- All fields returned in the projection (
{ field: 1, _id: 0 }) are part of that same index. The query engine resolves the request entirely within RAM from the B-tree index keys without examining or fetching any documents from disk (totalDocsExamined: 0).
Q15: What is the purpose of `$facet` in the aggregation framework?
Answer: $facet executes multiple parallel aggregation sub-pipelines on the exact same input document stream in a single query pass. It is the industry standard for creating faceted navigation, real-time analytics histograms, and paginated search results with total count metadata.
Q16: How does the Schema Versioning pattern eliminate database migration downtime?
Answer: Instead of locking database tables to add or rename columns, each document includes a schemaVersion: 2 integer. Application code contains adapter logic: when reading version 1 documents, it transforms them in-flight to version 2 structures, updating the document to version 2 on subsequent saves.
Q17: What is the difference between Read Concern `"local"` and `"majority"`?
Answer:
"local": Returns the most recent data on the queried node without checking whether other replica members have received it. Subject to rollback if the primary crashes before replicating."majority": Reads only data that has been acknowledged by a majority of voting nodes. The read view is immune to failover rollbacks.
Q18: What causes an in-memory `SORT` stage, and what is its memory limit?
Answer: If a query includes a .sort() clause that cannot use an existing B-tree index, MongoDB must load all candidate documents into RAM and sort them. Prior to MongoDB 6.0, this was limited to 100MB; exceeding it resulted in an error unless { allowDiskUse: true } was enabled.
Q19: What is a Partial Index, and why is it preferred over a Sparse Index?
Answer: A sparse index only indexes documents where the indexed field exists (even if null). A partial index allows any expressive filter expression (partialFilterExpression: { status: 'ACTIVE', score: { $gt: 50 } }), providing far greater flexibility, smaller index size, and better write performance.
Q20: What is a TTL Index, and what are its operational constraints?
Answer: TTL (Time-To-Live) indexes automatically remove documents after a set duration based on a date field. Constraints:
- Must be a single-field index on a
Datetype field. - Cannot be applied to compound indexes.
- Capped collections do not support TTL indexes.
- A background thread runs every 60 seconds to delete expired documents.
Q21: How do MongoDB Atlas Search and Lucene integration differ from standard `$text` indexes?
Answer: Standard $text indexes use basic stemmers and stop-word dictionaries stored in WiredTiger. Atlas Search embeds Apache Lucene directly within the mongot process, providing full fuzzy search, autocomplete, custom tokenizers, faceting, synonym mapping, and BM25 relevance scoring via the $search aggregation stage.
Q22: Explain the Outlier Pattern with a real-world example.
Answer: When 99.9% of documents have small arrays (e.g. Twitter users with < 500 followers) but 0.01% have millions of entries (celebrity accounts), embedding all followers into one document hits the 16MB BSON limit. The Outlier pattern sets a flag isOutlier: true on celebrity documents and redirects excess followers to an overflow collection.
Q23: What is the Bucket Pattern and why is it essential for IoT / Time-Series data?
Answer: Instead of inserting one document per telemetry reading (which creates billions of tiny documents and massive index overhead), the Bucket pattern groups readings for a given sensor into a single document representing a time window (e.g., 1 hour), holding an array of 60 readings with pre-aggregated min, max, and avg.
Q24: What is an Arbiter node and why should you avoid it in production?
Answer: An arbiter is a replica set member that participates in elections to break ties but stores no data. It should be avoided because:
- It cannot satisfy read operations.
- If an arbiter is present with a two-data-node set, losing one data node prevents satisfying majority write concerns (
w: "majority"). - Always deploy an odd number of data-bearing nodes instead.
Q25: What is Client-Side Field Level Encryption (CSFLE)?
Answer: CSFLE encrypts sensitive data (credit cards, SSNs) directly in the client application driver using a Customer Master Key (CMK) hosted in a KMS (AWS KMS, Azure Key Vault, HashiCorp Vault) before transmitting over the network. The MongoDB server stores and processes ciphertext only, preventing exposure even if database files or server memory are compromised.
Q26: What is the difference between `$push` and `$addToSet` in array updates?
Answer:
-
$push: Appends an item to an array regardless of whether it already exists (allows duplicates). -
$addToSet: Treats the array as a mathematical set, appending the value only if it does not already exist in the array.
Q27: How does `explain("executionStats")` reveal query performance bottlenecks?
Answer: Look for:
COLLSCAN(collection scan) vsIXSCAN(index scan).totalDocsExaminedvsnReturned: A high ratio indicates scanning non-matching documents.SORTstage: Indicates missing index for sorting.executionTimeMillis: Total server-side execution time.
Q28: How do Mongoose Pre and Post hooks differ?
Answer:
- Pre hooks (
schema.pre('save', ...)): Run before an operation executes. Ideal for sanitization, password hashing, or business logic validation. Can interrupt execution by throwing an error. - Post hooks (
schema.post('save', ...)): Run after an operation completes. Ideal for logging, audit trails, and triggering external asynchronous notifications.
Q29: What is the difference between Document Middleware and Query Middleware in Mongoose?
Answer:
- Document middleware (
save,validate,remove):thisrefers to the document instance being modified. - Query middleware (
find,findOneAndUpdate,updateMany):thisrefers to the Query object being executed, not the document itself.
Q30: How does Mongoose `populate()` work, and what is its performance implication?
Answer: populate() is an application-level join. Mongoose issues a secondary query (Model.find({ _id: { $in: ids } })) behind the scenes and stitches the returned documents in Node.js memory. In high-throughput systems, deep or chained populate() calls cause $lookup in aggregation pipelines for single-query database joins.
Q31: What is the difference between `$unwind` with and without `preserveNullAndEmptyArrays`?
Answer: By default, $unwind discards input documents if the array field is null, missing, or empty ([]). Setting preserveNullAndEmptyArrays: true ensures that documents without array elements are preserved in the pipeline output with null values.
Q32: What is Jumbo Chunk in a sharded cluster and how is it resolved?
Answer: A chunk is flagged as "jumbo" when it exceeds the maximum chunk size (default 64MB) and cannot be split because all documents share the exact same shard key value. It is resolved by refining the shard key with an additional suffix field to increase cardinality.
Q33: How does MongoDB handle Split-Brain scenarios during network partitions?
Answer: MongoDB requires a strict majority quorum (
Q34: What is Queryable Encryption (introduced in MongoDB 6.0)?
Answer: An industry-first cryptographic breakthrough that allows client applications to execute randomized equality and range queries on encrypted fields directly on the server without ever decrypting the data on the database host.
Q35: What is the difference between `$merge` and `$out` in aggregation pipelines?
Answer:
-
$out: Replaces the entire target collection with the pipeline results, dropping existing indexes and data. -
$merge: Incrementally merges pipeline output into an existing collection (supporting insert, update, replace, or keep), allowing on-demand materialized views without dropping collections.
Q36: Explain the difference between `findByIdAndUpdate()` and `save()` in Mongoose.
Answer:
findByIdAndUpdate(): Issues a directfindAndModifycommand to MongoDB. Bypasses document middleware (pre('save')) unless explicitly configured.save(): Loads the full document into memory, applies all schema validations and document pre/post hooks, and issues an atomic update.
Q37: What is Read Preference `nearest` and when should it be used?
Answer: nearest directs read queries to the replica set member with the lowest network latency (measured via periodic pings). It is ideal for geographically distributed read-heavy clusters where eventual consistency is acceptable.
Q38: What are Capped Collections and where are they used?
Answer: Fixed-size circular collections that overwrite the oldest documents when allocated space is full. They maintain insertion order on disk without requiring indexes, making them ideal for high-throughput logging, telemetry, and oplog replication.
Q39: How does the Positional Operator `$` work in array updates?
Answer: The $ operator represents the index of the first array element that matched the query condition:
db.students.updateOne({ _id: 1, 'grades.course': 'CS101' }, { $set: { 'grades.$.passed': true } })
Q40: How do you perform database migrations reliably in enterprise MongoDB?
Answer:
- Implement Schema Versioning (
schemaVersion: N). - Dual-write / Dual-read in application services.
- Run background idempotency workers in batches using cursor streams and bulk writes (
bulkWrite) with small chunks to avoid memory starvation and lock contention.
// Equality & Comparison
db.users.find({ age: { $gte: 21, $lte: 40 } });
// In Array & Logical OR
db.users.find({ $or: [{ department: 'Engineering' }, { salary: { $gt: 150000 } }] });
// Regular Expression & Case Insensitivity
db.users.find({ email: { $regex: '@enterprise\\.io$', $options: 'i' } });
// Array contains all
db.users.find({ tags: { $all: ['mongodb', 'typescript'] } });
// Array element match
db.orders.find({ items: { $elemMatch: { sku: 'A100', qty: { $gte: 5 } } } });// Set fields & update date
db.users.updateOne({ _id: id }, { $set: { status: 'ACTIVE' }, $currentDate: { updatedAt: true } });
// Atomic Increment
db.users.updateOne({ _id: id }, { $inc: { loginCount: 1 } });
// Add unique element to array
db.users.updateOne({ _id: id }, { $addToSet: { roles: 'ADMIN' } });
// Push with slice modifier (keep latest 50)
db.users.updateOne({ _id: id }, { $push: { logs: { $each: [newLog], $slice: -50 } } });
// Remove item from array
db.users.updateOne({ _id: id }, { $pull: { tags: 'deprecated' } });// Match -> Group -> Sort -> Limit
db.sales.aggregate([
{ $match: { status: 'COMPLETED' } },
{ $group: { _id: '$region', totalRevenue: { $sum: '$amount' }, avgOrder: { $avg: '$amount' } } },
{ $sort: { totalRevenue: -1 } },
{ $limit: 5 }
]);
// Lookup (Left Join)
db.orders.aggregate([
{
$lookup: {
from: 'users',
localField: 'userId',
foreignField: '_id',
as: 'customer'
}
},
{ $unwind: '$customer' }
]);// List indexes
db.users.getIndexes();
// Compound Index (ESR)
db.users.createIndex({ department: 1, salary: -1, age: 1 });
// Partial Index
db.users.createIndex({ email: 1 }, { unique: true, partialFilterExpression: { isActive: true } });
// TTL Index (1 hour expiry)
db.tokens.createIndex({ createdAt: 1 }, { expireAfterSeconds: 3600 });
// Explain Query Execution Stats
db.users.find({ department: 'Engineering' }).explain('executionStats');Contributions are warmly welcomed! Please follow these steps:
- Fork the repository and create a feature branch (
git checkout -b feat/aggregation-stage). - Run test suites to ensure 100% test pass rate (
npm test). - Ensure TypeScript builds without errors (
npm run build). - Submit a Pull Request following our Contributing Guidelines and Code of Conduct.
Created and maintained with precision by Manthan Ankolekar.
This project is licensed under the ISC License.