ClickHouse is a column-oriented analytical database with a large, fast-moving C++ codebase. The most productive way to read it is to follow one query through stable architectural boundaries rather than memorize class names from one release.

This tour focuses on the path from SQL to processors and from MergeTree parts to columnar chunks. Source paths and symbols are representative of the 2026 codebase; always use the tag matching the server you operate.

A map before the details

A simplified query path is:

client protocol / HTTP
    -> parse SQL into syntax structures
    -> analyze names, types, and expressions
    -> build and optimize a QueryPlan
    -> construct a QueryPipeline
    -> execute connected Processors over columnar Chunks
    -> encode the result in the requested format/protocol

The storage path for a typical MergeTree query is:

StorageMergeTree
    -> choose parts
    -> prune partitions and marks
    -> schedule read tasks
    -> read/decompress selected column streams
    -> apply PREWHERE / filters
    -> feed chunks into the pipeline

This separation matters. SQL analysis determines what the query means; planning chooses which steps implement it; pipeline construction connects executable processors; storage code decides which bytes must be read.

1. Entry points and parsing

Queries arrive through native TCP, HTTP, or other interfaces. Session context carries settings, access rights, current database, quotas, external tables, and query identifiers.

The parser produces an abstract syntax tree that represents syntax, not a fully resolved plan. At this stage, an identifier may still need catalog resolution, a literal may need type inference, and a function name may need overload selection.

When navigating the repository, start with:

  • src/Parsers/ for tokens, AST nodes, and statement parsers;
  • src/Interpreters/ and src/Analyzer/ for semantic interpretation;
  • src/Functions/ and factory code for function registration.

ClickHouse has evolved from older interpreter/analyzer paths toward a query-tree analyzer. Historical articles often mix both designs. Check server settings and the release tag before assuming which path is active for a query.

Parsing is not validation

The SQL text:

SELECT region, sum(amount)
FROM events
WHERE event_date >= today() - 7
GROUP BY region
ORDER BY sum(amount) DESC
LIMIT 20;

can be syntactically valid while still referencing a missing table, inaccessible column, incompatible type, or unknown function. Semantic analysis resolves those questions and produces typed expressions.

2. Analysis and expression DAGs

Database expression trees repeat work. If several outputs depend on the same subexpression, representing every occurrence as an independent tree can duplicate evaluation.

ClickHouse uses directed acyclic graph structures for expression actions in important planning/execution paths. A DAG can share common nodes, track required inputs and produced outputs, and order casts and function calls.

Analysis is responsible for more than name lookup:

  • resolving tables, columns, aliases, and functions;
  • deriving data types and inserting conversions;
  • validating grouping and aggregate semantics;
  • expanding wildcards and aliases;
  • identifying required source columns;
  • applying access control and row policies;
  • preparing subqueries, joins, and scalar expressions.

When debugging a surprising cast or missing column, inspect the analyzed plan rather than only the parser AST.

3. QueryPlan: steps before processors

The src/Processors/QueryPlan/ tree contains plan steps for reads, filters, expressions, aggregation, sorting, joins, unions, limits, and output preparation.

A plan step describes an operation and its input/output header. Optimizer passes can transform the plan before it becomes an executable pipeline. This is the right layer for questions such as:

  • Can a filter move closer to the read?
  • Can an input order satisfy ORDER BY without a full sort?
  • Can a projection replace the base table read?
  • Can a limit reduce work in an earlier step?
  • Where should partial and final aggregation happen?

Use EXPLAIN variants to connect SQL to these steps. EXPLAIN indexes = 1 is particularly useful for MergeTree pruning; pipeline-oriented explain output shows processor multiplicity and connections.

4. Processors, ports, chunks, and blocks

ClickHouse execution is vectorized: functions operate on columns containing many values rather than invoking a virtual expression interpreter once per row.

Several similarly named structures have different roles:

  • a Column stores values of one logical type, often with specialized encodings such as constants, nullable maps, arrays, or low-cardinality dictionaries;
  • a Block pairs columns with names and types and is widely used as a schema/header and data container at APIs;
  • a Chunk carries a set of columns and row count through processor ports, with the header known by the pipeline;
  • a Processor consumes and produces chunks through input/output ports.

Processor scheduling is demand- and readiness-driven. A processor reports whether it needs input, has output, is ready to run, or is finished. Executors schedule work across threads while respecting pipeline dependencies.

Vectorization does not mean every operator always executes a hand-written SIMD instruction. It means the execution unit is columnar and batch-oriented, which reduces per-row dispatch and gives compilers and specialized functions opportunities to use SIMD.

5. Storage engines and factories

ClickHouse exposes tables through storage-engine interfaces. StorageFactory maps engine names from CREATE TABLE to constructors. Different engines implement different capabilities: reading, writing, mutations, sampling, projections, distributed execution, or external-system access.

The MergeTree family is central for persistent analytical tables. Relevant code lives under:

  • src/Storages/MergeTree/;
  • src/Storages/StorageMergeTree.*;
  • src/Processors/QueryPlan/ReadFromMergeTree.*.

Do not infer behavior for every engine from MergeTree. A Memory table, Distributed table, Kafka engine, and external database connector have different consistency and pushdown behavior.

6. MergeTree parts are immutable units

Each insert produces one or more data parts. Within a part, rows are sorted by the table’s sorting key (ORDER BY). Parts are immutable after publication; background merges combine smaller parts into larger sorted parts.

Depending on part format and settings, column streams may be stored in separate files (wide parts) or together (compact parts). Compressed blocks and mark files let the reader seek into column streams without decompressing unrelated ranges.

Part immutability simplifies concurrent reads and crash recovery: a query can hold references to a stable set of parts while merges create replacements. Publication and removal still require metadata and lifecycle coordination so readers never observe half-built parts.

Merges do not universally deduplicate

A plain MergeTree merge combines sorted parts; it does not automatically decide that rows with equal keys are duplicates to remove. Specialized engines add merge semantics:

  • ReplacingMergeTree selects versions according to its rules;
  • CollapsingMergeTree and VersionedCollapsingMergeTree collapse sign/version pairs;
  • SummingMergeTree and AggregatingMergeTree combine values or states;
  • TTL rules and mutations may remove or transform rows.

Even with specialized engines, merge-time cleanup is asynchronous. Queries that require final collapsed semantics may use FINAL, with a potentially significant cost.

7. Granules, marks, and the sparse primary index

MergeTree’s primary index is sparse. It stores key information at granule boundaries rather than one entry per row. The default fixed granularity is commonly 8,192 rows, controlled by index_granularity; adaptive byte-based granularity can create smaller row counts for wide rows.

A mark records offsets needed to begin reading compressed column streams near a granule. A predicate on the sorting key can eliminate ranges of marks. The engine then reads whole selected granules, so an index condition that selects one row may still read its surrounding granule.

This is why sorting-key design is an I/O design:

  • columns used in selective range predicates should appear in useful order;
  • low-cardinality prefixes can help grouping/locality but may reduce pruning for later key fields;
  • the primary key expression determines the sparse index and must be compatible with the sorting order;
  • partitioning should eliminate coarse parts, not create millions of tiny partitions.

8. Layered pruning

A MergeTree read can narrow data at several levels:

  1. partition pruning removes parts whose partition key cannot match;
  2. primary-key analysis selects mark ranges inside remaining parts;
  3. data-skipping indexes eliminate granules when their summaries prove no match;
  4. projections may provide an alternate pre-sorted or pre-aggregated representation;
  5. PREWHERE reads filter columns first, then reads other columns only for surviving rows within selected ranges;
  6. ordinary filters remove remaining rows.

These mechanisms have different guarantees. A Bloom-filter skipping index can prove absence for some predicates; it is not a unique lookup index. PREWHERE reduces column I/O after range selection; it does not replace the primary index.

ReadFromMergeTree and the range/index analysis code are valuable reading because they connect query conditions to selected parts, ranges, marks, rows, read pools, and parallel-replica behavior.

9. Aggregation

For GROUP BY, ClickHouse maintains aggregate states in hash tables. The key layout and hash-table variant depend on the number and types of grouping keys. Aggregate function state may be allocated in arenas to reduce per-item allocation overhead.

At higher cardinality or scale, execution can:

  • use two-level hash tables to partition state;
  • aggregate in parallel and merge thread-local states;
  • spill external aggregation data when configured thresholds are reached;
  • send partial states from shards to an initiating node;
  • merge states using aggregate functions’ state/merge/finalize interfaces.

Memory use depends on groups and state size, not just input rows. uniqExact can grow with distinct values; approximate functions keep bounded or more compact states at the cost of error. Operators should set memory limits and choose aggregates with their state behavior in mind.

10. Sorting and limits

A full sort can dominate an analytical query. The planner looks for existing order from MergeTree reads and for opportunities to use partial sorting, top-N algorithms, or limit-aware execution.

If the requested order is a prefix-compatible direction of the table’s sorting key, a read-in-order plan may avoid a global sort. If a filter destroys or combines ordering, the optimization may no longer apply. EXPLAIN is more reliable than assuming an ORDER BY key automatically satisfies every query order.

11. Distributed queries

A Distributed table or cluster query introduces a coordinator/worker split. The initiating server rewrites and sends shard queries, receives streams, and performs remaining merge, aggregation, sorting, or limit work.

Moving partial aggregation to shards reduces network traffic, but global correctness still requires final merging by group key. LIMIT, DISTINCT, joins, and non-deterministic functions have rewrite rules and edge cases. Replica selection and consistency settings determine which replica state a distributed query sees.

The source tree around distributed query execution is best read alongside a concrete EXPLAIN and server trace logs. Otherwise transport, planning, and storage code can look like one undifferentiated system.

12. Mutations and lightweight deletes

Because parts are immutable, traditional mutations create new versions of affected parts. That can rewrite far more data than the changed rows. Lightweight-delete mechanisms record deletion state and let reads ignore rows before later cleanup, depending on feature and version.

The design trade-off is consistent: cheap append/publish paths defer consolidation to background work. Operators must budget merge and mutation I/O, watch part counts, and avoid update patterns that fight the engine’s append-oriented layout.

A repeatable source-reading workflow

1. Pin a release tag

Check out the exact server version. Master may contain renamed classes, new analyzers, or features absent in production.

2. Start with one query

Use a small table and one SELECT. Capture:

  • EXPLAIN plan output;
  • index/mark selection output;
  • pipeline output;
  • trace logs for the query ID;
  • system.query_log and relevant profile events.

3. Follow nouns from the plan

Search for the plan-step name, then its pipeline initialization method, then processors it creates. For storage reads, follow the storage object into ReadFromMergeTree and read-task generation.

4. Read headers before implementations

C++ headers often reveal ownership, invariants, and extension points faster than a large implementation file. Then read the method that appears in the trace or plan.

5. Validate a hypothesis experimentally

Change one table property—sorting key, granularity, projection, or filter—and compare selected marks, bytes read, processor graph, and elapsed time. Source understanding becomes reliable when it predicts observable behavior.

Stable lessons from a changing codebase

Class names evolve, but several ideas remain central:

  • columnar batches amortize interpretation and improve locality;
  • immutable sorted parts make writes cheap and shift work to merges;
  • sparse indexes prune granules rather than locate individual rows;
  • plans are transformed before becoming processor pipelines;
  • aggregate states enable parallel and distributed merging;
  • performance comes from avoiding bytes and work, not from one isolated “vectorized” loop.

The source is approachable when read as these interacting contracts. Begin with a query, identify the plan boundary, follow the selected storage path, and confirm every conclusion with EXPLAIN and measurements on the pinned release.

Sources and source map