fix(outputs.influxdb_v2): Correct calculation of amount of batches for concurrent writes #18014
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AndrewChubatiuk
changed the title
AndrewChubatiuk
changed the title
3 tasks
AndrewChubatiuk
force-pushed
the
influx-v2-write-concurrency-drops-metrics
branch
2 times, most recently
from
67e750f to
1f81ae2
Compare
AndrewChubatiuk
changed the title
telegraf-tiger
bot
added
area/influxdb
plugin/output
srebhan
reviewed
Nov 18, 2025
Member
srebhan
left a comment
srebhan
left a comment
There was a problem hiding this comment.
Thanks for catching this @AndrewChubatiuk! Only one request to move your testing into an own test-function to not distort the benchmarks!
srebhan
changed the title
AndrewChubatiuk
force-pushed
the
influx-v2-write-concurrency-drops-metrics
branch
from
1f81ae2 to
31eea20
Compare
srebhan
approved these changes
Nov 20, 2025
Member
srebhan
left a comment
srebhan
left a comment
There was a problem hiding this comment.
Thanks a lot @AndrewChubatiuk!
srebhan
added
the
ready for final review
skartikey
reviewed
Nov 20, 2025
Contributor
skartikey
left a comment
skartikey
left a comment
There was a problem hiding this comment.
@AndrewChubatiuk Thanks! A couple of minor comments.
AndrewChubatiuk
force-pushed
the
influx-v2-write-concurrency-drops-metrics
branch
from
31eea20 to
a912da5
Compare
Contributor
Author
|
@skartikey thanks for review |
srebhan
approved these changes
Nov 20, 2025
Contributor
|
Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip. 📦 Click here to get additional PR build artifactsArtifact URLs |
skartikey
approved these changes
Nov 20, 2025
Contributor
skartikey
left a comment
skartikey
left a comment
There was a problem hiding this comment.
@AndrewChubatiuk Thanks for the contribution!
AndrewChubatiuk
deleted the
influx-v2-write-concurrency-drops-metrics
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
in influxdb_v2 output with
concurrent_writes> 1 part of data may be droppedhere algorithm calculates a size of buffer dividing amount of metrics by workers count and increments by one if remainder is > 0, but this is not taken into account during calculation of batches count inside
createBatches.In current implementation with metrics slice of size
100kand concurrent_writes set to13telegraf drops7684items(100000 // ⌈100000/13⌉) * ⌈100000/13⌉= (100000//7693) * 7693 = 12 * 7693 = 92316Checklist