Skip to content

Perf: fast CursorValues compare for StringViewArray using inline_key_…#16630

Merged
alamb merged 10 commits intoapache:mainfrom
zhuqi-lucas:fast_sort_with_inlined_fast_key
Jul 4, 2025
Merged

Perf: fast CursorValues compare for StringViewArray using inline_key_…#16630
alamb merged 10 commits intoapache:mainfrom
zhuqi-lucas:fast_sort_with_inlined_fast_key

Conversation

@zhuqi-lucas
Copy link
Contributor

@zhuqi-lucas zhuqi-lucas commented Jun 30, 2025

…fast

Which issue does this PR close?

The arrow-rs compare improvement has been merged, we can apply it to CursorValues compare for StringViewArray in datafusion.

Related arrow-rs changes:
apache/arrow-rs#7748

Rationale for this change

What changes are included in this PR?

Optimizing the CursorValues compare for StringViewArray using fast inlined fast key, and compare performance.

Are these changes tested?

Yes

--------------------
Benchmark sort_tpch.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      main ┃ fast_sort_with_inlined_fast_key ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ Q1108.19 ms │                       106.83 ms │     no change │
│ Q2101.67 ms │                       102.40 ms │     no change │
│ Q3665.25 ms │                       657.33 ms │     no change │
│ Q4129.60 ms │                       128.60 ms │     no change │
│ Q5260.81 ms │                       259.19 ms │     no change │
│ Q6277.54 ms │                       276.67 ms │     no change │
│ Q7457.42 ms │                       445.38 ms │     no change │
│ Q8306.32 ms │                       304.63 ms │     no change │
│ Q9319.58 ms │                       315.52 ms │     no change │
│ Q10466.82 ms │                       469.02 ms │     no change │
│ Q11253.35 ms │                       205.67 ms │ +1.23x faster │
└──────────────┴───────────┴─────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                              ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (main)3346.55ms │
│ Total Time (fast_sort_with_inlined_fast_key)3271.23ms │
│ Average Time (main)304.23ms │
│ Average Time (fast_sort_with_inlined_fast_key)297.38ms │
│ Queries Faster1 │
│ Queries Slower0 │
│ Queries with No Change10 │
│ Queries with Failure0 │
└────────────────────────────────────────────────┴───────────┘

Are there any user-facing changes?

No

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Continue optimizing the CursorValues compare for StringViewArray

3 participants