SERVER-84018: multiPlannerFallbackEngaged Flag - Complete Guide

What This Feature Does

When a MongoDB admin configures query settings (e.g., “for this query shape, use index X”), but the specified index can’t be used (maybe it was dropped), the planner falls back to multi-planning (trying all available indexes).

Our change adds a boolean flag multiPlannerFallbackEngaged to the slow query log and database profiler so operators can see when this fallback happened. This is useful because the fallback adds planning overhead that could explain why a query was slow.

Key Concepts

Query Planner

The component that decides HOW to execute a query. Given a query like find({a: 1, b: 1}), the planner looks at available indexes and generates candidate execution plans.

Single-Planning vs Multi-Planning

Single plan: Only one viable plan exists. Use it directly.
Multi-planning: Multiple candidate plans exist. The system runs them briefly, measures performance, and picks the winner. More expensive but finds the best plan.

Query Settings

Server-level configuration that constrains the planner for specific query shapes. Example: “For queries shaped like find({status: ...}), only use the {status: 1} index.” Requires a replica set or sharded cluster.

The Fallback

Query settings say “use index X”
Planner tries to build a plan with only index X
Index X doesn’t work (dropped, can’t satisfy the query)
Planner throws NoQueryExecutionPlans
Catch block removes the constraint (IGNORE_QUERY_SETTINGS)
Planner retries with all available indexes (multi-planning)
Query succeeds, but with extra overhead

OpDebug

The debug/diagnostic info collected for a single database operation. Contains metrics like keysExamined, docsExamined, fromMultiPlanner, etc. Gets written to the slow query log and profiler.

AdditiveMetrics

A nested struct inside OpDebug that holds metrics designed to be accumulated across sub-operations (e.g., across shards). Accessed via getAdditiveMetrics() which goes through an indirection.

CurOp

Represents the “Current Operation”. Each operation has one CurOp which contains one OpDebug. Lives for the entire duration of the operation.

PlanSummaryStats

A temporary struct created during query execution. After execution, setPlanSummaryMetrics() copies these stats into AdditiveMetrics.

Slow Query Log vs Profiler

Slow query log: Text log lines in mongod.log. Written via OpDebug::report().
Database profiler: BSON documents in system.profile collection. Written via OpDebug::append(). Queryable.

Architecture: Query Execution Flow

flowchart TD
    A["User sends: db.orders.find({a: 1, b: 1})"] --> B["find_cmd.cpp\nCommand handler"]
    B --> C["get_executor.cpp\nCreates query executor"]
    C --> D["retryMakePlanner()\nget_executor_helpers.cpp"]
    
    D --> E{"makePlanner(params)\nTry to build a plan"}
    E -->|Success| F["Plan Executor\nRuns the winning plan"]
    E -->|Throws NoQueryExecutionPlans| G{"Query settings\nexist?"}
    
    G -->|No| H["Re-throw\nReal failure"]
    G -->|Yes, already ignored| H
    G -->|Yes, not yet ignored| I["SET multiPlannerFallbackEngaged = true\nSet IGNORE_QUERY_SETTINGS\nRetry planning"]
    
    I --> E
    
    F --> J["OpDebug\nStores all metrics"]
    J --> K["report()\nSlow query log"]
    J --> L["append()\nProfiler system.profile"]

    style I fill:#ff6b6b,color:#fff
    style K fill:#4ecdc4,color:#fff
    style L fill:#4ecdc4,color:#fff

The Fallback Decision

flowchart TD
    A["makePlanner() throws\nNoQueryExecutionPlans"] --> B{"hasQuerySettings?"}
    B -->|No| C["throw — real failure\nNo settings to remove"]
    B -->|Yes| D{"Already tried\nwithout settings?"}
    D -->|Yes| E["throw — real failure\nSettings weren't the problem"]
    D -->|No| F["This is the fallback!\nSettings caused the failure"]
    
    F --> G["Set multiPlannerFallbackEngaged = true"]
    G --> H["Set IGNORE_QUERY_SETTINGS"]
    H --> I["Regenerate planner params"]
    I --> J["Loop back and retry\nmakePlanner()"]
    J --> K["Succeeds with multi-planning\nUsing all available indexes"]

    style F fill:#ff6b6b,color:#fff
    style G fill:#ff6b6b,color:#fff
    style C fill:#95a5a6,color:#fff
    style E fill:#95a5a6,color:#fff
    style K fill:#2ecc71,color:#fff

Why The Flag Lives on OpDebug Directly

flowchart TD
    subgraph CurOp["CurOp (one per operation, never replaced)"]
        subgraph OpDebug["OpDebug"]
            FLAG["multiPlannerFallbackEngaged ✅\nSet here, read here\nNever gets replaced"]
            
            subgraph AM["AdditiveMetrics (via getAdditiveMetrics())"]
                FMP["fromMultiPlanner\nkeysExamined\ndocsExamined"]
                BADFLAG["multiPlannerFallbackEngaged ❌\nGets LOST because\ninstance can be replaced"]
            end
        end
    end

    SET["retryMakePlanner()\nsets the flag"] --> FLAG
    READ["report() / append()\nreads the flag"] --> FLAG

    style FLAG fill:#2ecc71,color:#fff
    style BADFLAG fill:#e74c3c,color:#fff
    style SET fill:#3498db,color:#fff
    style READ fill:#3498db,color:#fff

Data Flow: Planning to Output

sequenceDiagram
    participant FC as find_cmd.cpp
    participant RMP as retryMakePlanner()
    participant MP as makePlanner()
    participant OD as OpDebug
    participant LOG as Slow Query Log
    participant PROF as Profiler

    FC->>RMP: Call with planner params
    RMP->>MP: Try to build plan (with query settings)
    MP-->>RMP: Throws NoQueryExecutionPlans!
    
    Note over RMP: Check: settings exist?<br/>Check: already tried without?
    Note over RMP: YES settings exist,<br/>NO haven't tried without
    
    RMP->>OD: multiPlannerFallbackEngaged = true
    Note over RMP: Set IGNORE_QUERY_SETTINGS<br/>Regenerate params
    
    RMP->>MP: Retry (without settings)
    MP-->>RMP: Success! (multi-planning)
    RMP-->>FC: Return planner
    
    Note over FC: Execute the query...
    
    FC->>OD: setPlanSummaryMetrics()
    OD->>LOG: report() — includes flag
    OD->>PROF: append() — includes flag

Where The Code Lives

Files We Changed

File	Purpose	What we did
`src/mongo/db/op_debug.h:672-674`	OpDebug field declarations	Added `bool multiPlannerFallbackEngaged{false}`
`src/mongo/db/op_debug.cpp:341`	`report()` - slow query log	Added `OPDEBUG_TOATTR_HELP_BOOL_NAMED(...)`
`src/mongo/db/op_debug.cpp:624`	`append()` - profiler	Added `OPDEBUG_APPEND_BOOL2(...)`
`src/mongo/db/query/get_executor_helpers.cpp:124`	`retryMakePlanner()` catch block	Set the flag when fallback happens
`jstests/noPassthrough/query/query_settings_fallback_profiler.js`	Integration test	New file

File	What it does
`src/mongo/db/curop.h` / `curop.cpp`	CurOp - the current operation tracker
`src/mongo/db/commands/query_cmd/find_cmd.cpp`	The find command handler
`src/mongo/db/query/get_executor.cpp`	Creates executors, calls retryMakePlanner
`src/mongo/db/query/query_planner.cpp`	The actual query planner
`src/mongo/db/query/query_planner_params.h`	Planner config (IGNORE_QUERY_SETTINGS flag)
`src/mongo/db/query/plan_summary_stats.h`	PlanSummaryStats struct
`jstests/libs/mochalite.js`	Test framework (describe/it/before/after)
`jstests/libs/query/query_settings_utils.js`	Query settings test helpers
`jstests/libs/profiler.js`	`getLatestProfilerEntry()` helper

Pull Request

https://github.com/mongodb/mongo/pull/1639#discussion_r3019101956

How To Find Things In This Codebase

Strategy 1: Start from ticket keywords

The ticket mentions “query settings”, “fallback”, “slow query log”:

Search IGNORE_QUERY_SETTINGS → leads to get_executor_helpers.cpp (the fallback)
Search "slow query" → leads to curop.cpp (where the log is written)
Search fromMultiPlanner → shows the pattern for existing similar flags

Strategy 2: Follow an existing similar feature

fromMultiPlanner is the closest existing flag. Trace it:

Where declared? → op_debug.h
Where set? → setPlanSummaryMetrics() in op_debug.cpp
Where emitted? → report() and append() in op_debug.cpp
Where tested? → profile_find.js

Strategy 3: VSCode shortcuts

Shortcut	Action
`Ctrl+P`	Open any file by name
`Ctrl+Shift+F`	Search across all files
`Ctrl+G`	Go to a specific line number
`F12`	Go to definition of a symbol

Strategy 4: Codebase conventions

Pattern	Meaning
`jstests/`	JavaScript integration tests (run against a real server)
`*_test.cpp`	C++ unit tests
`jstests/noPassthrough/`	Tests that manage their own server
`jstests/core/`	Tests that run against an existing server
`LOGV2_DEBUG(id, level, ...)`	Debug log (prints at verbosity >= level)

What This Feature Does#

Key Concepts#

Query Planner#

Single-Planning vs Multi-Planning#

Query Settings#

The Fallback#

OpDebug#

AdditiveMetrics#

CurOp#

PlanSummaryStats#

Slow Query Log vs Profiler#

Architecture: Query Execution Flow#

The Fallback Decision#

Why The Flag Lives on OpDebug Directly#

Data Flow: Planning to Output#

Where The Code Lives#

Files We Changed#

Related Files#

Pull Request#

How To Find Things In This Codebase#

Strategy 1: Start from ticket keywords#

Strategy 2: Follow an existing similar feature#

Strategy 3: VSCode shortcuts#

Strategy 4: Codebase conventions#