Skip to content

24.8.14 Backport of #84770: Handle NULLs in ALTER MODIFY COLUMN#1370

Merged
zvonand merged 2 commits intocustomizations/24.8.14from
backports/24.8.14/84770
Feb 7, 2026
Merged

24.8.14 Backport of #84770: Handle NULLs in ALTER MODIFY COLUMN#1370
zvonand merged 2 commits intocustomizations/24.8.14from
backports/24.8.14/84770

Conversation

@zvonand
Copy link
Collaborator

@zvonand zvonand commented Feb 5, 2026

Changelog category (leave one):

  • Backward Incompatible Change

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

ALTER MODIFY COLUMN now requires explicit DEFAULT when converting nullable columns to non-nullable types. Previously such ALTERs could get stuck with cannot convert null to not null errors, now NULLs are replaced with column's default expression. (ClickHouse#84770 by @vdimir)

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Performance tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All with Aarch64
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • Tiered Storage (2h)

…umn_null_to_default

Handle NULLs in ALTER MODIFY COLUMN
@zvonand zvonand added the 24.8.14 label Feb 5, 2026
@altinity-robot
Copy link
Collaborator

altinity-robot commented Feb 5, 2026

This is an automated comment for commit c8d0661 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Check nameDescriptionStatus
Sign aarch64There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS❌ error
Sign releaseThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS❌ error
Successful checks
Check nameDescriptionStatus
AST fuzzerRuns randomly generated queries to catch program errors. The build type is optionally given in parenthesis. If it fails, ask a maintainer for help✅ success
BuildsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Compatibility checkChecks that clickhouse binary runs on distributions with old libc versions. If it fails, ask a maintainer for help✅ success
Docker keeper imageThe check to build and optionally push the mentioned image to docker hub✅ success
Docker server imageThe check to build and optionally push the mentioned image to docker hub✅ success
Grype Scan altinityinfra/clickhouse-keeper:1370-24.8.14.10505.altinitytestThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Grype Scan altinityinfra/clickhouse-server:1370-24.8.14.10505.altinitytest-alpineThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Grype Scan altinityinfra/clickhouse-server:1370-24.8.14.10505.altinitytestThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Install packagesChecks that the built packages are installable in a clear environment✅ success
Integration testsThe integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests✅ success
Ready for releaseThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Alter attach partition 1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Alter attach partition 2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Alter move partitionThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Alter replace partitionThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Benchmark aws_s3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Benchmark gcsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Benchmark minioThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Clickhouse Keeper no_ssl 1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Clickhouse Keeper no_ssl 2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Clickhouse Keeper ssl 1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Clickhouse Keeper ssl 2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 LDAP authenticationThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 LDAP external_user_directoryThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 LDAP role_mappingThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Parquet aws_s3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Parquet minioThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 ParquetThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 aws_s3-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 aws_s3-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 azure-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 azure-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 gcs-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 gcs-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 minio-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 minio-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 S3 minio-3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Tiered Storage minioThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Tiered Storage s3amazonThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 Tiered Storage s3gcsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 aes_encryptionThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 aggregate_functions-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 aggregate_functions-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 aggregate_functions-3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 atomic_insertThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 base_58There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 data_typesThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 datetime64_extended_rangeThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 dnsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 enginesThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 exampleThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 extended_precision_data_typesThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 functionsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 kafkaThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 kerberosThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 key_valueThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 lightweight_deleteThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 memoryThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 part_moves_between_shardsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 selectsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 session_timezoneThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 ssl_server-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 ssl_server-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 ssl_server-3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 tiered_storageThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 versionThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression aarch64 window_functionsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Alter attach partition 1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Alter move partitionThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Benchmark aws_s3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Benchmark gcsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Benchmark minioThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Clickhouse Keeper no_ssl 1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Clickhouse Keeper no_ssl 2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Clickhouse Keeper ssl 1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Clickhouse Keeper ssl 2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release LDAP authenticationThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release LDAP external_user_directoryThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release LDAP role_mappingThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release Parquet aws_s3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release ParquetThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release S3 azure-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release S3 gcs-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release S3 minio-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release S3 minio-3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release aggregate_functions-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release aggregate_functions-2There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release aggregate_functions-3There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release atomic_insertThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release base_58There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release data_typesThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release datetime64_extended_rangeThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release dnsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release enginesThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release exampleThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release extended_precision_data_typesThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release functionsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release kafkaThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release kerberosThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release key_valueThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release memoryThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release part_moves_between_shardsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release selectsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release session_timezoneThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release ssl_server-1There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release versionThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Regression release window_functionsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Stateful testsRuns stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc✅ success
Stateless testsRuns stateless functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc✅ success
Stress testRuns stateless functional tests concurrently from several clients to detect concurrency-related errors✅ success

@zvonand
Copy link
Collaborator Author

zvonand commented Feb 7, 2026

Code from upstream was merged clearly. The PR contains a test that passes.

@zvonand zvonand merged commit b63d329 into customizations/24.8.14 Feb 7, 2026
345 of 348 checks passed
@Selfeer
Copy link
Collaborator

Selfeer commented Mar 17, 2026

PR #1370 Audit Review

AI audit note: This review was generated by AI (gpt-5.3-codex).

Audit update for PR #1370 (24.8.14 Backport of ClickHouse#84770: Handle NULLs in ALTER MODIFY COLUMN)


Confirmed defects

Medium: Shared default-expression AST is reused without cloning in conversion pipeline

  • Impact: Conversion actions for nullable->non-nullable paths can mutate/shared-reparent metadata-owned AST nodes, creating thread-safety risk and nondeterministic behavior under concurrent reads/mutations on the same table.
  • Anchor: src/Interpreters/inplaceBlockConversions.cpp / convertRequiredExpressions().
  • Trigger: A table has a default expression for a column being converted from nullable to non-nullable; conversion path uses that default expression while multiple conversion builds happen over shared metadata snapshots.
  • Affected transition: ALTER MODIFY COLUMN metadata defaults -> convertRequiredExpressions() AST build -> ExpressionAnalyzer actions DAG build.
  • Why defect: The new path directly reuses it->second.expression instead of cloning, while the same module explicitly documents that expressions must be cloned before passing into analysis/execution stages.
  • Smallest logical reproduction: (1) create table with default expression on nullable column; (2) run nullable->non-nullable modify path that triggers conversion while concurrent reads/mutation processing use same metadata defaults; (3) observe shared AST reuse through conversion expression construction.
  • Fix direction (short): Clone metadata default AST before embedding into conversion expression (it->second.expression->clone()).
  • Regression test direction (short): Add a stress/integration test that repeatedly triggers conversion-expression construction with defaults under concurrency and assert stable behavior/no AST corruption.
  • Affected subsystem / blast radius: In-place block conversion used by MergeTree read/mutation/projection conversion checks; impacts correctness/reliability during ALTER-driven type transitions.
        /// Converting a column from nullable to non-nullable may cause 'Cannot convert column' error when NULL values exist.
        /// Users should specify DEFAULT expression in ALTER MODIFY COLUMN statement to replace NULL values.
        if (isNullableOrLowCardinalityNullable(column_in_block.type) && !isNullableOrLowCardinalityNullable(required_column.type))
        {
            ASTPtr default_value;
            if (auto it = column_defaults.find(required_column.name); it != column_defaults.end())
                default_value = it->second.expression;
            ...
            auto convert_func = makeASTFunction("_CAST",
                makeASTFunction("ifNull", std::make_shared<ASTIdentifier>(required_column.name), default_value),
                std::make_shared<ASTLiteral>(required_column.type->getName()));
    if (column_default)
    {
        /// expressions must be cloned to prevent modification by the ExpressionAnalyzer
        auto column_default_expr = column_default->expression->clone();

Coverage summary

  • Scope reviewed: PR 24.8.14 Backport of #84770: Handle NULLs in ALTER MODIFY COLUMN #1370 deltas in conversion logic (inplaceBlockConversions.*), ALTER pre-check path (MergeTreeData::checkAlterIsPossible), conversion callsites (IMergeTreeReader::performRequiredConversions, AlterCommands::apply), and stateless tests (03575_*, 02662_*).
  • Categories failed: C++ memory/lifetime/thread-safety (shared AST ownership in conversion build path).
  • Categories passed: Call-graph and transition coverage for entrypoints in ALTER validation, read-time conversion, projection compatibility checks; logical branch coverage for nullable->non-nullable handling (explicit default, fallback default, forbid path), error-contract checks (BAD_ARGUMENTS/conversion failures), rollback/partial-update reasoning (no new multi-step state mutation introduced), concurrency checks found no lock-order additions/deadlock paths in touched code, integer/signedness/resource-leak/UB-cast categories in changed lines.
  • Assumptions/limits: Static audit only (no runtime execution); conclusions are bounded to PR 24.8.14 Backport of #84770: Handle NULLs in ALTER MODIFY COLUMN #1370 diff and directly connected call paths in current branch snapshot.

@Selfeer
Copy link
Collaborator

Selfeer commented Mar 17, 2026

PR #1370 CI Triage

Summary

Category Count Details
PR-caused regression 0 None
Pre-existing flaky tests 1 01509_check_many_parallel_quorum_inserts_long
Infrastructure issues 3 Sign aarch64, Sign release, Integration test timeouts (first run)
Cascade failures 0 None

Verdict: PASS — No failures caused by this PR.

The CI report explicitly confirms: New Fails in PR: 0, Regression New Fails: 0, Checks New Fails: 0 (compared with base sha c4a9654b209941568095d648270f8874ad97692d).

Detailed Analysis

Pre-existing Flaky Tests (Unrelated)

Test Check First Run Rerun History
01509_check_many_parallel_quorum_inserts_long Stateless tests (release) FAIL (2026-02-05) OK (2026-02-06) Also failed on PR=0 master (2025-10-24, ASAN)

This is a known flaky test related to parallel quorum insert timing. It failed on the first run and passed on the rerun, confirming it is not related to this PR's changes.

Infrastructure Issues (Unrelated)

Sign aarch64 / Sign release — Persistent CI Infrastructure Failure

  • Status: error on both runs (2026-02-05 and 2026-02-06)
  • Error: Error: Unknown, exit_code 0
  • Evidence: The same Sign failures occur across every recent PR in the CI database: PRs 1524, 1525, 1532, 1533, 1534, 1540, and PR=0 (master). This is a systemic CI infrastructure issue, completely unrelated to any PR content.

Integration Tests Timeout — Resolved on Rerun

  • First run (2026-02-05): All 10 integration test jobs (6 aarch64 + 4 release) hit "Job Timeout Expired," causing ~30+ individual test ERRORs across multiple test suites.
  • Rerun (2026-02-06): All 10 integration test jobs passed cleanly with 0 failures:
    • aarch64: 443 + 404 + 359 + 419 + 621 + 461 = 2,707 passed
    • release: 778 + 776 + 718 + 758 = 3,030 passed
  • Transient infrastructure timeout, not related to this PR.

Recommendations

  1. No action required for this PR — all failures are unrelated to the change.
  2. Sign job infrastructure issue should be investigated separately as it affects all PRs system-wide.
  3. The flaky test 01509_check_many_parallel_quorum_inserts_long has a history of intermittent failures and should be tracked for potential stabilization.

@Selfeer
Copy link
Collaborator

Selfeer commented Mar 17, 2026

PR #1370 Audit Review

AI audit note: This review was generated by AI (gpt-5.3-codex).

Audit update for PR #1370 (24.8.14 Backport of ClickHouse#84770: Handle NULLs in ALTER MODIFY COLUMN)

Confirmed defects

Medium: Shared default-expression AST is reused without cloning in conversion pipeline

* **Impact:** Conversion actions for nullable->non-nullable paths can mutate/shared-reparent metadata-owned AST nodes, creating thread-safety risk and nondeterministic behavior under concurrent reads/mutations on the same table.

* **Anchor:** `src/Interpreters/inplaceBlockConversions.cpp` / `convertRequiredExpressions()`.

* **Trigger:** A table has a default expression for a column being converted from nullable to non-nullable; conversion path uses that default expression while multiple conversion builds happen over shared metadata snapshots.

* **Affected transition:** `ALTER MODIFY COLUMN` metadata defaults -> `convertRequiredExpressions()` AST build -> `ExpressionAnalyzer` actions DAG build.

* **Why defect:** The new path directly reuses `it->second.expression` instead of cloning, while the same module explicitly documents that expressions must be cloned before passing into analysis/execution stages.

* **Smallest logical reproduction:** (1) create table with default expression on nullable column; (2) run nullable->non-nullable modify path that triggers conversion while concurrent reads/mutation processing use same metadata defaults; (3) observe shared AST reuse through conversion expression construction.

* **Fix direction (short):** Clone metadata default AST before embedding into conversion expression (`it->second.expression->clone()`).

* **Regression test direction (short):** Add a stress/integration test that repeatedly triggers conversion-expression construction with defaults under concurrency and assert stable behavior/no AST corruption.

* **Affected subsystem / blast radius:** In-place block conversion used by MergeTree read/mutation/projection conversion checks; impacts correctness/reliability during ALTER-driven type transitions.
        /// Converting a column from nullable to non-nullable may cause 'Cannot convert column' error when NULL values exist.
        /// Users should specify DEFAULT expression in ALTER MODIFY COLUMN statement to replace NULL values.
        if (isNullableOrLowCardinalityNullable(column_in_block.type) && !isNullableOrLowCardinalityNullable(required_column.type))
        {
            ASTPtr default_value;
            if (auto it = column_defaults.find(required_column.name); it != column_defaults.end())
                default_value = it->second.expression;
            ...
            auto convert_func = makeASTFunction("_CAST",
                makeASTFunction("ifNull", std::make_shared<ASTIdentifier>(required_column.name), default_value),
                std::make_shared<ASTLiteral>(required_column.type->getName()));
    if (column_default)
    {
        /// expressions must be cloned to prevent modification by the ExpressionAnalyzer
        auto column_default_expr = column_default->expression->clone();

Coverage summary

* **Scope reviewed:** PR [24.8.14 Backport of #84770: Handle NULLs in ALTER MODIFY COLUMN #1370](https://github.com/Altinity/ClickHouse/pull/1370) deltas in conversion logic (`inplaceBlockConversions.*`), ALTER pre-check path (`MergeTreeData::checkAlterIsPossible`), conversion callsites (`IMergeTreeReader::performRequiredConversions`, `AlterCommands::apply`), and stateless tests (`03575_*`, `02662_*`).

* **Categories failed:** C++ memory/lifetime/thread-safety (shared AST ownership in conversion build path).

* **Categories passed:** Call-graph and transition coverage for entrypoints in ALTER validation, read-time conversion, projection compatibility checks; logical branch coverage for nullable->non-nullable handling (explicit default, fallback default, forbid path), error-contract checks (`BAD_ARGUMENTS`/conversion failures), rollback/partial-update reasoning (no new multi-step state mutation introduced), concurrency checks found no lock-order additions/deadlock paths in touched code, integer/signedness/resource-leak/UB-cast categories in changed lines.

* **Assumptions/limits:** Static audit only (no runtime execution); conclusions are bounded to PR [24.8.14 Backport of #84770: Handle NULLs in ALTER MODIFY COLUMN #1370](https://github.com/Altinity/ClickHouse/pull/1370) diff and directly connected call paths in current branch snapshot.

@zvonand can you please check the audit review, if it makes sense to raise an issue for this?

@zvonand
Copy link
Collaborator Author

zvonand commented Mar 17, 2026

This is only a backport, almost 1:1 matches upstream's code

In backports, we take the upstream PRs as is

@Selfeer
Copy link
Collaborator

Selfeer commented Mar 18, 2026

@zvonand can we at least just make sure there is nothing critical in the audit review? I can mark it as verified afterwards.

@zvonand
Copy link
Collaborator Author

zvonand commented Mar 19, 2026

I do not see anything critical. Again, this is only a backport, we take it as is.

@Selfeer Selfeer added the verified Verified by QA label Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants