Skip to content

feature: flow-php/arrow-ext#2268

Open
norberttech wants to merge 9 commits into1.xfrom
rs-arrow-extension
Open

feature: flow-php/arrow-ext#2268
norberttech wants to merge 9 commits into1.xfrom
rs-arrow-extension

Conversation

@norberttech
Copy link
Member

Change Log


Added

  • arrow-ext and expose parquet reader/writer
  • ParquetEngine abstraction into flow-php/parquet

Fixed

Changed

  • Let flow-php/parquet use engines based on arrow extension availability

Removed

Deprecated

Security

Heads Up - I'm not rust developer and I don't have solid rust experience, building this extension was a learning process inspired by @ptondereau

Reviews, especially the rust part are welcome!

To use extension in nix shell just use nix-shell which has argument with-arrow-ext set by default to true.
To develop the extension (compile it and run tests) use nix-shell --arg with-rust=true

Introduce arrow-ext and expose parquet reader/writer.
Introduce ParquetEngine abstraction into flow-php/parquet.
Make flow-php/parquet use engines based on arrow extension availability.
@norberttech norberttech added this to the 0.35.0 milestone Mar 24, 2026
@norberttech norberttech force-pushed the rs-arrow-extension branch 5 times, most recently from b831844 to e0280f0 Compare March 25, 2026 10:29
@codecov
Copy link

codecov bot commented Mar 25, 2026

Codecov Report

❌ Patch coverage is 80.76923% with 115 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.19%. Comparing base (6bec09d) to head (8c786df).
⚠️ Report is 3 commits behind head on 1.x.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##              1.x    #2268      +/-   ##
==========================================
+ Coverage   82.88%   83.19%   +0.30%     
==========================================
  Files        1361     1368       +7     
  Lines       50797    51220     +423     
==========================================
+ Hits        42104    42612     +508     
+ Misses       8693     8608      -85     
Components Coverage Δ
etl 90.61% <ø> (+0.83%) ⬆️
cli 85.76% <ø> (ø)
lib-array-dot 94.82% <ø> (ø)
lib-azure-sdk 60.01% <ø> (ø)
lib-doctrine-dbal-bulk 95.16% <ø> (ø)
lib-filesystem 83.33% <73.33%> (-0.10%) ⬇️
lib-types 88.88% <ø> (+1.50%) ⬆️
lib-parquet 70.18% <81.42%> (+1.30%) ⬆️
lib-parquet-viewer 83.04% <ø> (ø)
lib-snappy 90.65% <ø> (+0.93%) ⬆️
lib-dremel 0.00% <ø> (ø)
lib-postgresql 88.15% <ø> (ø)
lib-telemetry 89.57% <ø> (ø)
bridge-filesystem-async-aws 90.97% <ø> (ø)
bridge-filesystem-azure 89.43% <ø> (ø)
bridge-monolog-http 97.45% <ø> (ø)
bridge-openapi-specification 91.50% <ø> (ø)
symfony-http-foundation 74.11% <ø> (ø)
bridge-psr18-telemetry 100.00% <ø> (ø)
bridge-psr7-telemetry 100.00% <ø> (ø)
bridge-telemetry-otlp 89.29% <ø> (+0.25%) ⬆️
bridge-symfony-http-foundation-telemetry 88.23% <ø> (ø)
bridge-symfony-telemetry-bundle 67.60% <ø> (ø)
adapter-chartjs 84.72% <ø> (ø)
adapter-csv 88.65% <ø> (ø)
adapter-doctrine 92.00% <ø> (ø)
adapter-elasticsearch 92.78% <ø> (ø)
adapter-google-sheet 99.02% <ø> (ø)
adapter-http 67.91% <ø> (ø)
adapter-json 87.61% <ø> (ø)
adapter-logger 47.36% <ø> (ø)
adapter-parquet 78.89% <42.85%> (-0.41%) ⬇️
adapter-text 81.25% <ø> (ø)
adapter-xml 82.07% <ø> (ø)
adapter-avro 0.00% <ø> (ø)
adapter-excel 94.02% <ø> (ø)
adapter-postgresql 0.00% <ø> (ø)
bridge-phpunit-telemetry 48.14% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

@ptondereau ptondereau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mostly good here! Great work!
I left some comments. Please don't hesitate to acknowledge

@norberttech
Copy link
Member Author

@ptondereau I believe it should be good now, thanks for your help! If you want to take another look, I'm probably going to aim to merge this one tomorrow

@ptondereau
Copy link

LGTM!

allow to upload partial coverages from different workflows using flags
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

2 participants