Skip to content

How It Works

django-query-doctor uses a four-stage pipeline to intercept, identify, analyze, and report on query performance issues. This page explains each stage and how they connect.

The Four-Stage Pipeline

flowchart LR
    A["INTERCEPT\n(middleware / decorator)"] --> B["FINGERPRINT\n(normalize + hash)"]
    B --> C["ANALYZE\n(detect issues)"]
    C --> D["REPORT\n(output results)"]

    style A fill:#e3f2fd,stroke:#1565c0
    style B fill:#f3e5f5,stroke:#7b1fa2
    style C fill:#fff3e0,stroke:#e65100
    style D fill:#e8f5e9,stroke:#2e7d32

Stage 1: Intercept

django-query-doctor hooks into Django's database layer using connection.execute_wrapper(). This is a low-level mechanism that wraps every SQL call at the database connection level, not at the ORM or model layer. This means:

  • It captures raw SQL regardless of how it was generated (ORM, raw queries, third-party libraries).
  • It works without DEBUG=True (unlike connection.queries).
  • It captures the full stack trace for each query, allowing precise file:line attribution.

The interceptor is installed by the middleware at the start of each request and removed at the end. You can also install it manually using the @diagnose decorator or the diagnose_queries() context manager.

Key design decision: We intentionally use execute_wrapper instead of connection.queries because the latter requires DEBUG=True and does not capture stack traces. Our approach works in production and staging environments without configuration changes.

Stage 2: Fingerprint

Every captured SQL query is normalized and hashed to produce a fingerprint:

  1. Normalization -- Literal values (WHERE id = 42) are replaced with placeholders (WHERE id = ?). IN clauses are collapsed (IN (?, ?, ?) becomes IN (?+)).
  2. Hashing -- The normalized SQL string is hashed with SHA-256 to produce a compact, deterministic identifier.
  3. Grouping -- Queries with the same fingerprint are grouped together. A group of 50 queries with the same fingerprint is a strong signal of an N+1 problem.

This fingerprint-based approach is what distinguishes django-query-doctor from simple query counters. Two queries that differ only in parameter values are recognized as the "same" query.

Stage 3: Analyze

Grouped queries are passed through a chain of analyzers, each responsible for detecting a single category of issue. Every analyzer receives the full list of captured queries (with fingerprints, stack traces, and timing data) and returns zero or more Prescriptions.

Analyzer Detects Guide
NPlusOneAnalyzer N+1 query patterns from FK/M2M traversal N+1 Queries
DuplicateAnalyzer Exact and near-duplicate queries within one request Duplicates
MissingIndexAnalyzer WHERE/ORDER BY on columns without indexes Missing Indexes
FatSelectAnalyzer SELECT * when only a few columns are used Fat SELECT
QuerysetEvalAnalyzer Unnecessary queryset evaluations (e.g., len(qs) instead of qs.count(), if qs: instead of qs.exists()) Queryset Evaluation
DRFSerializerAnalyzer N+1 patterns inside DRF serializers DRF Serializer
QueryComplexityAnalyzer Overly complex queries (too many JOINs, subqueries) Query Complexity

Analyzers are independent and stateless. You can enable or disable each one individually via settings.

Stage 4: Report

Prescriptions from all analyzers are collected and passed to one or more reporters. Reporters format the results for different consumption targets:

Reporter Output
ConsoleReporter Rich-formatted terminal output (falls back to plain text if Rich is not installed)
JsonReporter Structured JSON for CI/CD pipelines and tooling
HtmlReporter Self-contained HTML page for the admin dashboard
LogReporter Python logging integration
OtelReporter OpenTelemetry spans and attributes

What Is a Prescription?

Every issue detected by an analyzer is returned as a Prescription dataclass. This is the core data structure of django-query-doctor. A Prescription contains:

Field Type Description
issue_type IssueType enum The category of issue (e.g., N_PLUS_ONE, DUPLICATE_QUERY, MISSING_INDEX)
severity Severity enum One of CRITICAL, WARNING, INFO
description str Human-readable description of the problem
fix_suggestion str A ready-to-apply code fix (e.g., Add .select_related('author') to your queryset)
callsite CallSite Source file path, line number, and function name where the issue originates
query_count int Number of queries involved in this issue
time_saved_ms float Estimated time savings if the fix is applied
fingerprint str The SHA-256 fingerprint of the query group
extra dict Additional metadata (e.g., table name, field name)

Prescriptions are not just warnings. They are actionable: the fix_suggestion field contains the exact code change you need to make, and the callsite field tells you exactly where to make it.

Connection-Level, Not ORM-Level

A common question is whether django-query-doctor works with raw SQL or only with ORM queries. Because interception happens at the database connection level (connection.execute_wrapper), it captures all SQL that passes through Django's database backend:

  • ORM queries (Model.objects.filter(...))
  • Raw queries (connection.cursor().execute(...))
  • Queries from third-party packages (django-rest-framework, django-filter, etc.)
  • Queries from Django internals (sessions, auth, content types)

The stack tracer then maps each query back to your source code, filtering out Django internals and third-party code to show you only the lines you control.

Data Flow Diagram

flowchart TD
    REQ["HTTP Request"] --> MW["QueryDoctorMiddleware"]
    MW --> EW["connection.execute_wrapper()"]
    EW --> INT["Interceptor captures SQL + stack trace"]
    INT --> FP["Fingerprinter normalizes + hashes"]
    FP --> AG["Group by fingerprint"]
    AG --> A1["NPlusOneAnalyzer"]
    AG --> A2["DuplicateAnalyzer"]
    AG --> A3["MissingIndexAnalyzer"]
    AG --> A4["...other analyzers"]
    A1 --> RX["Prescriptions"]
    A2 --> RX
    A3 --> RX
    A4 --> RX
    RX --> R1["ConsoleReporter"]
    RX --> R2["JsonReporter"]
    RX --> R3["HtmlReporter"]
    R1 --> OUT["Terminal / Logs / Dashboard"]
    R2 --> OUT
    R3 --> OUT

Further Reading