Skip to content
/ server Public

MDEV-13594 (was MDEV-18530): Implement -> and ->> JSON path operators#4711

Open
kjarir wants to merge 2 commits intoMariaDB:mainfrom
kjarir:mdev-18530-json-operators
Open

MDEV-13594 (was MDEV-18530): Implement -> and ->> JSON path operators#4711
kjarir wants to merge 2 commits intoMariaDB:mainfrom
kjarir:mdev-18530-json-operators

Conversation

@kjarir
Copy link

@kjarir kjarir commented Feb 28, 2026

Summary:
Adds support for MySQL-compatible JSON path operators '->' and '->>' as aliases for JSON_EXTRACT and JSON_UNQUOTE(JSON_EXTRACT).

Key Changes:

  • Lexer: Modified MINUS_OR_COMMENT in sql_lex.cc to detect '->' and '->>'.
  • Parser: Restructured primary_expr in sql_yacc.yy to support operator chaining without shift/reduce conflicts.
  • Symbols: Added JSON_ARROW_SYM and JSON_UNQUOTED_ARROW_SYM to lex.h.
  • Tests: Added comprehensive verification for literal/column extraction, chaining, and usage in VIRTUAL columns with INDEX in mysql-test/main/json_operators.test.
    Labels: gsoc26

@CLAassistant
Copy link

CLAassistant commented Feb 28, 2026

CLA assistant check
All committers have signed the CLA.

@kjarir
Copy link
Author

kjarir commented Feb 28, 2026

Fixed a syntax error in the newly added test file mysql-test/main/json_operators.test where some comments were being misinterpreted by the mysqltest runner. All CI checks should now progress correctly.

@kjarir
Copy link
Author

kjarir commented Feb 28, 2026

I've reviewed the CI logs and found that the Buildbots and AppVeyor were failing due to a mismatch in the sys_vars.ft_boolean_syntax_basic test result. Because the lexer now parses -> as a single token (JSON_ARROW_SYM), a test case that assigned an unquoted full-text boolean syntax + -><...; threw a slightly different syntax error (near '-><' instead of near '><'). I have updated the ft_boolean_syntax_basic.result file to reflect this new, expected behavior and pushed the fix. The CI should now pass successfully.

Copy link
Member

@grooverdan grooverdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done on implementation with a good start to tests.

Do you know where these operators show up in terms of https://mariadb.com/docs/server/reference/sql-structure/operators/operator-precedence ? Can there be test cases around validate they stay in that order (if possible)?

Do the results of these show up the same results as the MySQL implementation? (mysql-test/suite/json/inc/json_functions.inc). If you include these into MariaDB, adapting to its form (so maybe not an include if not needed), in their own commit. Include authorship and credit to the original Oracle authors and include what changes you made also in commit message.

@@ -0,0 +1,64 @@
--source include/have_innodb.inc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't use or implement any specific innodb features so this can be removed.

The MDEV header below should be echo statements.
MDEV-18530 is a MDEV closed as a duplicate therefore using the MDEV-13594 MDEV is recommented.

After adding a test case like this, mtr --record and then commit the json_operators.result file in the same commit.


--echo # 5. Edge Cases: Invalid JSON and Paths
--echo # Native functions return NULL or error; operators should match
SELECT '{"a": 1'->'$.a';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as these may be ERROR conditions there will be a --error before the failing SQL statements with the error they genenrate.

SELECT '{"a": 1}'->'invalid_path';

--echo # 6. Integration with Generated Columns
--echo # This is a common use case for these operators in MySQL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't really need to say "in MySQL"

--echo # If the string from ->> is valid JSON, another -> can follow.
SELECT '{"outer": "{\\"inner\\": 1}"}'->>'$.outer'->'$.inner' AS complex_chain;

# End of MDEV-18530 tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The final statement in a test case is"

--echo End of 13.0 tests

For the purpose of ease of merging. The end of a particular MDEV's tests becomes obvious with the beginning of the next MDEV.

if (unlikely(list == NULL) ||
unlikely(list->push_back($1, thd->mem_root)) ||
unlikely(list->push_back($3, thd->mem_root)))
MYSQL_YYABORT;
Copy link
Member

@grooverdan grooverdan Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a list memory allocation to both a local variable, that if one of the two push_backs fail, becomes a leak. The other allocations have this assigned to a lex structure of $$ at the time MYSQL_YYABORT is called.

@gkodinov gkodinov added the External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements. label Mar 2, 2026
@kjarir kjarir force-pushed the mdev-18530-json-operators branch 3 times, most recently from 7800009 to ad067f1 Compare March 2, 2026 12:03
@kjarir kjarir requested a review from grooverdan March 2, 2026 15:09
@grooverdan grooverdan changed the title MDEV-18530: Implement -> and ->> JSON path operators MDEV-13594 (was MDEV-18530): Implement -> and ->> JSON path operators Mar 3, 2026
SELECT f1->>NULL FROM t1_mysql;

--echo # Invalid path returns a warning
--echo error ER_INVALID_JSON_PATH
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So its technically not a ER_INVALID_JSON_PATH.

JSON_EXTRACT is documented to returning NULL and warnings on invalid paths, so its consistent.

@kjarir kjarir force-pushed the mdev-18530-json-operators branch from 117708c to da52af6 Compare March 4, 2026 04:24
@grooverdan grooverdan self-requested a review March 5, 2026 02:50
@SORVER
Copy link

SORVER commented Mar 5, 2026

@grooverdan
Hey everyone, I thought this was a GSoC 2026 project. Should I start working on it for GSoC, or is it now done?

@grooverdan
Copy link
Member

@grooverdan Hey everyone, I thought this was a GSoC 2026 project. Should I start working on it for GSoC, or is it now done?

It was intended to be, but its pretty much done, so there's no point starting on it now.

Let's keep discussion of GSoC, any anything not related with the review of this PR on https://mariadb.zulipchat.com.

There are other opportunities, discuss on the zulip chat to work out a mutual interest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements.

Development

Successfully merging this pull request may close these issues.

6 participants