Repository navigation
[BUG] 5.4-SNAPSHOT : ClickHouse : asterisk column modifiers fail or mis-parse (APPLY as alias, docs chain, STRICT, EXCEPT regex) #2636
Description
AI disclosure: this issue was prepared with AI coding agents, revised and verified by me.
Failing SQL Feature:
ClickHouse column modifiers on * / t.*: the documented APPLY modifier is silently mis-parsed as a select-item alias, the documented modifier chain from the SELECT docs fails with a ParseException, and the STRICT variants plus the regex-string form of EXCEPT (implemented in the ClickHouse grammar, covered by its stateless tests) are unsupported.
SQL Example:
-- 1) documented in the ClickHouse SELECT docs, mis-parsed on master:
-- parses as AllColumns with alias "APPLY(sum)"
SELECT * APPLY(sum) FROM t;
-- 2) the modifier chain exactly as shown in the ClickHouse docs, fails:
SELECT * REPLACE(i + 1 AS i) EXCEPT (j) APPLY(sum) FROM columns_transformers;
-- 3) STRICT variants (undocumented, but in the ClickHouse grammar and
-- its stateless tests) and the regex form of EXCEPT, all fail:
SELECT * EXCEPT STRICT (a, b) FROM t;
SELECT * REPLACE STRICT (1 AS a, 2 AS b) FROM t;
SELECT * EXCEPT '^tmp_' FROM t;Baseline that already parses: SELECT * EXCEPT (a, b) FROM t, SELECT * REPLACE (a AS b) FROM t, SELECT * EXCLUDE (a) FROM t.
Software Information:
- JSqlParser version: current master 5.4-SNAPSHOT, commit 02b9d94 (tested 2026-09-14)
- Database: ClickHouse
Additional context
- ClickHouse grammar accepts
STRICTafter both EXCEPT and REPLACE, and a string-literal regex pattern for EXCEPT (ExpressionElementParsers.cpp:is_strict,ParserStringLiteral->ASTColumnsExceptTransformer::setPattern, formatted asEXCEPT STRICTinASTColumnsTransformers.cpp); covered by the stateless tests01470_columns_transformers.sqland02343_analyzer_column_transformers_strict.sql. - The SELECT modifiers documentation lists only APPLY, EXCEPT and REPLACE; the STRICT and regex forms are undocumented.
- References: ExpressionElementParsers.cpp, ASTColumnsTransformers.cpp
- Related: PR feat: support ClickHouse COLUMNS(...) APPLY/EXCEPT/REPLACE transformers (#2631) #2635 adds the
COLUMNS('regexp')context for the documented modifiers; the asterisk position is untouched there, so this issue is independent of it.
Activity
kyy-logs commentedon Sep 19, 2026
I'd like to take this one.
Proposed scope — only the first two items, to keep the patch reviewable:
SELECT * APPLY(sum): parse asAllColumns+ColumnsTransformerinstead of falling into the alias path (APPLYis inNonReservedWord(), so it currently becomes the alias"APPLY(sum)").* EXCEPT (a) REPLACE (b AS c)).I'd leave
STRICTand the regex form ofEXCEPTfor a follow-up — they look like separate grammar work, and I'd rather not bundle four behaviours into one diff.Approach: extend the
AllColumns(boolean)production inJSqlParserCC.jjt(~L7946) to accept a repeatedColumnsTransformer(), reuse the existingColumnsExpression/ColumnsTransformerclasses, then updateAllColumns, theExpressionVisitor/DeParserpair, and add round-trip tests viaassertSqlCanBeParsedAndDeparsed().Heads-up: #2652 also touches
AllColumns.java— happy to rebase after it lands, or hold off if you'd prefer.AI disclosure: I use an AI coding assistant; I review and test everything I submit and take responsibility for it.
kyy-logs commentedon Sep 19, 2026
Follow-up on the approach — one thing I'd like to get right before writing code.
AllColumnscurrently models these modifiers itself (exceptColumns/replaceExpressions/exceptKeyword), whileColumnsTransformer(from #2635) models the same keywords for theCOLUMNS(...)path. If I giveAllColumnsaList<ColumnsTransformer>, the same syntax ends up with two representations.Which would you prefer?
ColumnsTransformer—AllColumnsholds aList<ColumnsTransformer>and the three legacy fields go away. Consistent withColumnsExpression, and chains like* EXCEPT (a) REPLACE (b AS c)fall out of the repeated-transformer grammar for free. Cost: it's a breaking API change, andAllTableColumns extends AllColumnsso its constructors change too. Internal usage looks small —AllTableColumnsreadsgetExceptKeyword(), plus the deparser and one test.EXCEPT/REPLACE.I lean (1) since 5.4 is still a snapshot, but I don't want to churn public API without a nod.
One more question: should
* APPLY(...)andtable.* APPLY(...)both be supported? The issue only shows the plain*form.